Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Compile times
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
gaarde
n00b
n00b


Joined: 05 Jul 2002
Posts: 3

PostPosted: Fri Jul 05, 2002 8:35 pm    Post subject: Compile times Reply with quote

To give gentoo users an idea of what to expect, I would like to see if we could collect the times it takes to bootstrap and / or compile popular packages on various platforms.

These would only be estimations due to 1) the inaccuracy of the collection method 2) the number of variations on hardware configurations (speed of harddrives, bus speed, amount of ram, etc) and 3) Is that a 700mhz AMD, Intel, PPC, SPARC, or (insert arch here)?

More (highly inaccurate) examples:

Bootstrapping on a 100mhz CPU is estimated to take 20 to 30 hours.
Bootstrapping on a 1000mhz CPU is estimated to take 2 to 3 hours.
KDE3 takes 1.5 to 4 hours on a 350 mhz CPU.
GNOME2 takes 1 to 3 hours on a 500 mhz CPU.
Back to top
View user's profile Send private message
delta407
Bodhisattva
Bodhisattva


Joined: 23 Apr 2002
Posts: 2876
Location: Chicago, IL

PostPosted: Sat Jul 06, 2002 12:41 am    Post subject: Re: Compile times Reply with quote

gaarde wrote:
KDE3 takes 1.5 to 4 hours on a 350 mhz CPU.
GNOME2 takes 1 to 3 hours on a 500 mhz CPU.


Yeah, you wish. :D

A rough guesstimate might be nice in the docs, but this won't work as a feature in Portage for the reasons outlined in this thread.
_________________
I don't believe in witty sigs.
Back to top
View user's profile Send private message
digitalnick
Apprentice
Apprentice


Joined: 30 Jun 2002
Posts: 243
Location: Lawrence KS USA

PostPosted: Sun Jul 07, 2002 4:44 pm    Post subject: Reply with quote

preaultec in #gentoo is working a logging script that displays wall times for builds ect .... if people start using it accurate times can be guestamated a little better.

http://12.224.215.99/misc/lec there is the link give it a shot its pretty nice.
Back to top
View user's profile Send private message
gaarde
n00b
n00b


Joined: 05 Jul 2002
Posts: 3

PostPosted: Tue Aug 06, 2002 3:32 am    Post subject: P2-350, 128M, UDMA2 Reply with quote

Bootstrap started: Mon Aug 5 20:09:28 UTC 2002
Bootstrap stopped: Tue Aug 6 03:14:50
Back to top
View user's profile Send private message
gaarde
n00b
n00b


Joined: 05 Jul 2002
Posts: 3

PostPosted: Tue Aug 06, 2002 2:50 pm    Post subject: P2-350, 128M, UDMA2 Reply with quote

I thought I had an times for emerge system... but somehow... they are missing... And note, vim was not compiling anything for gvim (USE="-X -gnome -gtk -gtk2")

ccache started: Tue Aug 6 03:37:28
ccache stopped: Tue Aug 6 03:38:01

vim started: Tue Aug 6 14:37:59
vim stopped: Tue Aug 6 14:48:17
Back to top
View user's profile Send private message
phong
Bodhisattva
Bodhisattva


Joined: 16 Jul 2002
Posts: 778
Location: Michigan - 15 & Ryan

PostPosted: Fri Sep 20, 2002 1:38 am    Post subject: Reply with quote

To pick up this discussion, I was thinking it might be a good idea to make some sort of time-logging feature or wrapper for portage, and then combine that with gentoo-stats to get ideas of typical compile times for various packages. I mentioned that here. "How long will it take on my box?" is a common question, and there usually aren't hard numbers so I think it would be a good idea. So...
Bad idea?
Good idea?
Comments?
_________________
"An empty head is not really empty; it is stuffed with rubbish. Hence the difficulty of forcing anything into an empty head."
-- Eric Hoffer
Back to top
View user's profile Send private message
boyo
n00b
n00b


Joined: 02 Nov 2002
Posts: 53
Location: MI

PostPosted: Wed Nov 06, 2002 1:26 am    Post subject: Reply with quote

You can use the time command and have it write the results to a file or just display the results to stdout when the timed command (ie emerge) finishes.

Code:
time -o textfilename emerge ebuild1 ebuild2 ... ebuildN
to save to a file

or

Code:
time emerge ebuildname
for screen output
Back to top
View user's profile Send private message
squanto
Guru
Guru


Joined: 20 Apr 2002
Posts: 524
Location: Rochester, NY, USA

PostPosted: Wed Nov 06, 2002 5:25 am    Post subject: Reply with quote

I second phong's idea, that would be cool, and we could set it up with the bb accounts so that you could say what your username is in your make.conf and then it could be a part of rsync..... wait, that would be like .......
huh? what did I just say?
Back to top
View user's profile Send private message
mooman
Apprentice
Apprentice


Joined: 06 Nov 2002
Posts: 175
Location: Vancouver, WA

PostPosted: Wed Nov 06, 2002 8:45 pm    Post subject: Reply with quote

I think it would be nice to get some timers or loggers in there.

Right now my method of timing is:
10 pm - start a build (like stage 1 or 2), go to bed
8 am - check to see if it's done before leaving for work. Nope? Geez!
7 pm - check to see if it's done after coming home from work. Yep? Finally!

If something goes wrong, repeat process...

So my method is accurate to about +/- 11 hours.

:wink:
_________________
Linux user off and on since circa 1995
Back to top
View user's profile Send private message
Incon
n00b
n00b


Joined: 03 Sep 2002
Posts: 10

PostPosted: Thu Nov 07, 2002 1:27 am    Post subject: Reply with quote

I think it would be good to have a ratings system rather than a time.
This way we don't have to be too precise and don't have to collect system-specific info.

Off the top of my head you could have a scale of 1-10 where 1 is extracting a binary package like setiathome-bin and a 10 is compiling Openoffice.

For extra geekness the scale could be logarithmic, so that the difference between and 3 and 4 really does matter.
But the problem with having a scale that actually means something, be it linear or logarithmic or monkeys in a barrel, is that you then need to have a unit to measure against and in that case is it really that much different than measuring time on different systems?

Guess that's why I suggested the old 1-10 scale first.
Damn random tangents.
Back to top
View user's profile Send private message
guero61
l33t
l33t


Joined: 14 Oct 2002
Posts: 811
Location: Behind you

PostPosted: Thu Nov 07, 2002 1:40 am    Post subject: Reply with quote

I'll say it again -- LFS has a great idea of "bash units". Here's the explanation from book 4.0:

Quote:

SBUs are Static Bash Units and they are our way of identifying how long a package takes to compile. Why don't we use normal times like anybody else?

The biggest problem is that times cannot be accurate, not even a little bit. So many people install LFS on so many different systems, the times it takes to compile something varies too much. One package may take 20 minutes on one system, but that same package may take 3 days on another (this is not an exaggeration). So instead we've come up with a Static Bash Unit or SBU.

It works like this: the very first package you compile in this book is Bash in Chapter 5 and it'll be statically linked. The time it takes to compile this package will be the basis and called the SBU. All other compile times are relative to the time it takes to install Bash. For example, GCC-3.2 takes about 9.5 SBUs and it's proven that this number is fairly consistent among a lot of different systems. So multiply 9.5 by the number of seconds it takes for Bash to install (the SBU value) and you get a close approximation of how long GCC will take on your system.

Note: We've seen that SBUs don't work well on SMP based machines. So all bets are off if you're lucky enough to have an SMP setup.


I think it's a good idea; it's relative to the machine, yet makes a common denominator that can be easily built into an ebuild for time estimation. "SBU=0.659", and from past compilations, emerge could easily compute actual install time.
Back to top
View user's profile Send private message
gsfgf
Veteran
Veteran


Joined: 08 May 2002
Posts: 1266

PostPosted: Thu Nov 07, 2002 3:58 am    Post subject: Reply with quote

guero61 wrote:


I think it's a good idea; it's relative to the machine, yet makes a common denominator that can be easily built into an ebuild for time estimation. "SBU=0.659", and from past compilations, emerge could easily compute actual install time.


yea, emerge could even store the time it takes to build bash. Another idea for gentooer.
_________________
Aim:gsfgf0
Back to top
View user's profile Send private message
guero61
l33t
l33t


Joined: 14 Oct 2002
Posts: 811
Location: Behind you

PostPosted: Thu Nov 07, 2002 4:47 am    Post subject: Reply with quote

I posted a request...

https://bugs.gentoo.org/show_bug.cgi?id=10365

I have this eternal fear of looking entirely stupid to those great minds behind great linux distros like Gentoo. Hopefully it wasn't too dumb.
Back to top
View user's profile Send private message
squanto
Guru
Guru


Joined: 20 Apr 2002
Posts: 524
Location: Rochester, NY, USA

PostPosted: Thu Nov 07, 2002 2:11 pm    Post subject: Reply with quote

guero61 wrote:
I posted a request...

https://bugs.gentoo.org/show_bug.cgi?id=10365

I have this eternal fear of looking entirely stupid to those great minds behind great linux distros like Gentoo. Hopefully it wasn't too dumb.


I think it is well worded, and even if they don't decide to use that idea, I don't think they will laugh at you. I am sure even developers want to see how long it will take to compile things.
Back to top
View user's profile Send private message
masseya
Bodhisattva
Bodhisattva


Joined: 17 Apr 2002
Posts: 2602
Location: Baltimore, MD

PostPosted: Thu Nov 07, 2002 6:10 pm    Post subject: Reply with quote

squanto wrote:
I think it is well worded, and even if they don't decide to use that idea, I don't think they will laugh at you. I am sure even developers want to see how long it will take to compile things.

I just wanted to back squanto's play. (yeah, i've been waiting to use that phrase.. :)) It's very well worded and I think it's a good idea. Then again, I'm not in a position to do anything about it as I have no more power than your average billy goat in the decisions made by the gentoo developers.
_________________
if i never try anything, i never learn anything..
if i never take a risk, i stay where i am..
Back to top
View user's profile Send private message
guero61
l33t
l33t


Joined: 14 Oct 2002
Posts: 811
Location: Behind you

PostPosted: Thu Nov 07, 2002 10:18 pm    Post subject: Reply with quote

w00t! I just realized I now have the designation, "Tux's lil' helper"!!! I HAVE ASCENDED!!! :lol:













:oops:
Back to top
View user's profile Send private message
metalhedd
l33t
l33t


Joined: 30 May 2002
Posts: 692
Location: Ontario Canada

PostPosted: Thu Nov 07, 2002 11:34 pm    Post subject: Reply with quote

That SBU Thing is a terrific Idea. just felt the need to comment, quite frankly I dont care how inaccurate it is, as long as it can tell me if it its gonna be close to 10 minutes than 10 hours.
Back to top
View user's profile Send private message
masseya
Bodhisattva
Bodhisattva


Joined: 17 Apr 2002
Posts: 2602
Location: Baltimore, MD

PostPosted: Fri Dec 06, 2002 10:34 pm    Post subject: Reply with quote

There was another thread that was started on bash units which would be nice to merge here since this was the thread that mentioned them first. However, I'll simply provide a link to the other thread which is now in the duplicates forum.

bash units?
_________________
if i never try anything, i never learn anything..
if i never take a risk, i stay where i am..
Back to top
View user's profile Send private message
MrOnion
n00b
n00b


Joined: 17 Nov 2002
Posts: 24

PostPosted: Tue Dec 17, 2002 9:36 am    Post subject: Reply with quote

Is anything actually being done about this? I was just wondering because i have some free time on my hands and I honestly dont see why this isnt do able. Something like:

Download bash
get the time1 in seconds
untar bash
./configure --(options i know not of yet)
make
get time2 in seconds
time2-time1 = your static bash unit

Then of course some sorta database would have to be made which stored peoples compile times for each program in SBUs (compile time/ SBU)

So has ne one heard any progress on this? If not i will start scripting right away.

Ryan Jones
Back to top
View user's profile Send private message
MrOnion
n00b
n00b


Joined: 17 Nov 2002
Posts: 24

PostPosted: Tue Dec 17, 2002 11:29 am    Post subject: Reply with quote

Ok well nobody replied and i got bored so heres what i've come up with.

www.z-lan.com/gentoostuff/SBUfinder

That will give u an SBUtime in seconds.

If that was written to a filethen it could be referenced whenever needed.

Now i realise an SBU unit is not much use on its own. So either SBUs for each program would have to be collected through gentoo-stats, or would have to be entered into a webpage which could either be referenced on the net with a search for a package name, or a text file with the data (packagename/SBUs)downloaded and then referenced with a shell script which would basically just grep the text file.

Possible problem i have noticed:
What if the version of bash changes, either the same version of bash will have to be kept or all SBUs will have to be updated.

TODO: ?
1. Perhaps run the compile a few times so that the data is more reliable.
2. Incluse the untaring in the SBU.
3. Time the download to give people an idea ofhow long that will take.
4. clean up after the SBUfinder has been run.

Any input would be useful.

Ryan Jones

On a side note. when running that script on my AthlonXP1600, 512DDR,
On gentoo my SBU was 82
On slackware 86

So all this optimising stuff doesnt seem up to that much. (but portage undoubtably rules because of its dependancy handling.)
Back to top
View user's profile Send private message
guero61
l33t
l33t


Joined: 14 Oct 2002
Posts: 811
Location: Behind you

PostPosted: Tue Dec 17, 2002 5:44 pm    Post subject: Reply with quote

If you read my bug submission, it covers the questions you have. DRobbins has moved it from "NEW" to "ASSIGNED", so I'm not sure if they're working on it or have just acknowledged it's presence. Now that the semester is over, I'll likely spend a bit of time and come up with a functional method.


Patience, sir Onion.
Back to top
View user's profile Send private message
nngs
n00b
n00b


Joined: 01 Oct 2002
Posts: 48

PostPosted: Tue Dec 17, 2002 7:05 pm    Post subject: my results Reply with quote

for DELL inspiron 2650c
cpu celeron-M 1.5G, 256M ddr mem, 256M SWAP
CHOST="i686..."
CFLAGS="-march=pentium4 -O3 -pipe"
bootstraping time 142m (i wish i could separate the downloading time and the compiling time, but i cann't)
kernel compiling 7m21s

for my old P166
cpu pentium (no mmx) 166MHz, 48M EDO 60ns mem, 128M SWAP
CHOST="i585..."
CFLAGS="-march=pentium -O3 -pipe -fomit-frame-pointer"
bootstraping time (stage1 -> stage2) 1321m31s
emerge system (stage2 -> stage3) 609m28s
kernel compiling time 68m45s
Back to top
View user's profile Send private message
metalhedd
l33t
l33t


Joined: 30 May 2002
Posts: 692
Location: Ontario Canada

PostPosted: Wed Jan 08, 2003 6:11 am    Post subject: Reply with quote

the only problem I see, is that Bash isn't installed by default on EVERY OS. it will work out well for gentoo, and all linux distro's in general, but not BSDs, or other unix variants. but i cant think of anything that is.
Back to top
View user's profile Send private message
Princess Firefly
Tux's lil' helper
Tux's lil' helper


Joined: 21 Apr 2002
Posts: 80

PostPosted: Fri Mar 07, 2003 4:50 am    Post subject: progress indicator, not time indicator Reply with quote

I got my other post moved to the duplicate threads forum maybe with reason, but here's an idea that might be easier to implement albeit the info wouldn't be as nice as something like a coordinated SBU thing.

It seems like there must be some way to parse through the make files and figure out how many gcc statements are going to be needed for a certain compile. Then, while a program is compiling we could have a little progress indicator that could say "12 out of 100 statements done" or "~12% done" or something. Of course the hard time will vary greatly between machines and between packages, it would give some sort of indication of when a build would complete and allow a user to roughly gauge the progress. If it said 24 out of 5000 and it was completing one compile every 10 seconds, you could turn off the monitor and go to bed. Also, this wouldn't require any sort of data gathering or coordination, it'd be totally localized. Finally, if it worked, it'd probably be possible to roughly extrapolate the time remaining based on the speed of compiliation up to a certain point (like every 20 gcc commands or something). Sort of like ETA's for internet downloads.

Is that totally impossible? Totally stupid? What do you think?
Back to top
View user's profile Send private message
noff
Guru
Guru


Joined: 11 Nov 2002
Posts: 388
Location: College Park, Maryland

PostPosted: Fri Mar 07, 2003 7:12 am    Post subject: Reply with quote

Princess Firefly, that isn't a bad idea. I don't care about absolute time I just want relative. I want to know whether to go to bed or hang around to five minutes. If it is accurate to within 30% I would be happy. It is a lot better than the current method of know for some packages but not others. It is easy to find out how long kde is taking people to compile, but how about mysql or any other 99% of the packages in portage. If you figure out how to do it please implement it. I would use it.
_________________
What Larry was saying is that if you make it too easy for programmers, then poor programmers will be able to do things best left to good programmers, and will inevitably do them poorly. Everyone will suffer in the long term as a result." - Tom Chance
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum