| View previous topic :: View next topic |
| Author |
Message |
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
 Posted: Sat Jun 22, 2002 4:26 am Post subject: Speed up recompilation with ccache |
 |
|
Update: from portage-2.0.7 up, CCACHE_DIR must be set as /var/tmp/ccache instead of /var/cache/ccache.
From the ccache ebuild:
| Quote: | | Description: ccache is a fast compiler cache. It is used as a front end to your compiler to safely cache compilation output. When the same code is compiled again the cached output is used giving a significant speedup. |
From the ccache manual page:
| Quote: | INSTALLATION
There are two ways to use ccache. You can either prefix your compile commands with "ccache" or you can create a symbolic link between ccache and the names of your compilers. The first method is most convenient if you just want to try out ccache or wish to use it for some specific projects. The second method is most useful for when you wish to use ccache for all your compiles.
To install for usage by the first method just copy ccache to somewhere in your path.
To install for the second method do something like this:
| Code: | cp ccache /usr/local/bin/
ln -s /usr/local/bin/ccache /usr/local/bin/gcc
ln -s /usr/local/bin/ccache /usr/local/bin/cc |
This will work as long as /usr/local/bin comes before the path to gcc (which is usually in /usr/bin).
(...)
HOW IT WORKS
The basic idea is to detect when you are compiling exactly the same code a 2nd time and use the previously compiled output.
(...) |
These are the steps to use ccache in Gentoo:
1. Update Portage, if needed. You'll need Portage version 2.0.2 or later (this is a sandbox issue).
2. Install ccache. (It's worth noticing that ccache has no dependencies, so it can be installed quite early, even before the "emerge system" install step! I wouldn't recommend installing it before bootstraping, though.)
3. Read the ccache manual.
4. Create the symbolic links:
| Code: | # ln -s /usr/bin/ccache /bin/gcc
# ln -s /usr/bin/ccache /bin/cc
# ln -s /usr/bin/ccache /bin/g++
# ln -s /usr/bin/ccache /bin/c++ |
5. Create the ccache cache dir:
| Code: | | # mkdir /var/tmp/ccache |
6. Add the following line to /etc/profile:
| Code: | | export CCACHE_DIR="/var/tmp/ccache" |
7. Log out and back in, or manually run
| Code: | | # export CCACHE_DIR="/var/tmp/ccache" |
8. Be happy!
I made some tests to see if it is really worthy. I ran the following commands twice with ccache disabled and three times with ccache enabled:
| Code: | # ebuild /usr/portage/net-www/mozilla/mozilla-1.0-r1.ebuild clean
# ebuild /usr/portage/net-www/mozilla/mozilla-1.0-r1.ebuild unpack
# time ebuild /usr/portage/net-www/mozilla/mozilla-1.0-r1.ebuild compile |
The results were, in order:
| Code: | real 36m47.153s
user 32m57.496s
sys 2m18.362s
real 36m45.588s
user 32m59.537s
sys 2m17.597s
real 38m5.326s
user 33m10.850s
sys 2m22.291s
real 10m54.726s
user 6m1.358s
sys 1m37.028s
real 10m30.090s
user 5m42.772s
sys 1m37.005s |
You can draw your own conclusions... 
_________________
| www.gentoo.org | www.tldp.org | www.google.com |
Last edited by fghellar on Sun Jul 21, 2002 4:05 pm; edited 1 time in total |
|
| Back to top |
|
 |
lamer
n00b
Joined: 16 May 2002
Posts: 16
Location: St. Paul, Mn
|
| Posted: Sat Jun 22, 2002 3:59 pm Post subject: dude |
|
|
| what are you compiling this on that mozilla only takes 38 minutes? |
|
| Back to top |
|
|
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
|
| Back to top |
|
|
asimon
l33t
Joined: 27 Jun 2002
Posts: 979
Location: Germany, Old Europe
|
| Posted: Thu Jun 27, 2002 3:08 pm Post subject: |
|
|
Do those ccache timings with newer KDE sources and gcc 3.1 and you'll get completly different numbers 
I tested ccache for compiling cvs snapshots of kde 3.0.5 with gcc 3.1.
99% of all items in ccache's statistics were: "Compiler option not supported".
The compiler option in question was some obscure pre-processor flag which KDE uses when it detects gcc 3. So ccache was useless for me.
Cheers,
Andreas |
|
| Back to top |
|
|
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
| Posted: Thu Jun 27, 2002 7:23 pm Post subject: |
|
|
Can you provide more information about this? For example, what do you mean with "obscure pre-processor flag"? Did you link the gcc 3 binaries to ccache (they get different names if gcc 2 is installed too)? etc...
_________________
| www.gentoo.org | www.tldp.org | www.google.com | |
|
| Back to top |
|
|
rukhoven
n00b
Joined: 18 Apr 2002
Posts: 54
Location: The flat lands of nether elevation
|
| Posted: Thu Jun 27, 2002 7:31 pm Post subject: |
|
|
If you want to compile something with other cflags, then it will use your previously compiled code however wont it? It seems it will not be usefull in such a case...
_________________
*****************************
AMDXp 2000 per sis745 con 60 GB HD
......Is what I'd like to have....... |
|
| Back to top |
|
|
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
| Posted: Thu Jun 27, 2002 7:44 pm Post subject: |
|
|
That would be a really stupid program, don't you think?
Anyways, step 3 above should make it clear to you, esp. the section "How it works". (You can find it by going to Google and typing "ccache manual", if you don't have it installed.)
_________________
| www.gentoo.org | www.tldp.org | www.google.com | |
|
| Back to top |
|
|
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
|
| Back to top |
|
|
asimon
l33t
Joined: 27 Jun 2002
Posts: 979
Location: Germany, Old Europe
|
| Posted: Thu Jun 27, 2002 8:30 pm Post subject: |
|
|
| fghellar wrote: | | Can you provide more information about this? For example, what do you mean with "obscure pre-processor flag"? Did you link the gcc 3 binaries to ccache (they get different names if gcc 2 is installed too)? etc... |
I always got "argument -MT is unsupported" from ccache. This options gets added by KDEs building system when you use gcc 3. When ccache encountered this option in a compile command (and that was the case 99%) the compilation output was not put into the cache. After compiling most of KDEs modules (build dir > 2.5 GB) the cache of ccache was only 20 MB big, so it was quite useless. After two more compilations of the whole kde stuff I only had around 100 cache hits. But it was great when I used gcc 2.96 to compile KDE.
After some compilations you can check with ccache -s how efficent the cache was.
Cheers,
Andreas |
|
| Back to top |
|
|
rukhoven
n00b
Joined: 18 Apr 2002
Posts: 54
Location: The flat lands of nether elevation
|
| Posted: Thu Jun 27, 2002 8:42 pm Post subject: |
|
|
| fghellar wrote: | That would be a really stupid program, don't you think?
Anyways, step 3 above should make it clear to you, esp. the section "How it works". (You can find it by going to Google and typing "ccache manual", if you don't have it installed.) |
So when I change the CFLAGS, it will not use the cached stuff if I recompile a program? Ok you seem sure of that, but I couldn't interpret the manual in such way to rule that out/in ... Well I guess I'll just try that. Would some code be compiled more than once when compiling say, kdelibs ?
_________________
*****************************
AMDXp 2000 per sis745 con 60 GB HD
......Is what I'd like to have....... |
|
| Back to top |
|
|
asimon
l33t
Joined: 27 Jun 2002
Posts: 979
Location: Germany, Old Europe
|
| Posted: Thu Jun 27, 2002 8:57 pm Post subject: |
|
|
| Quote: |
So when I change the CFLAGS, it will not use the cached stuff if I recompile a program? If ccache supports the flags which get used it works. Usually the options you put in CFLAGS are no problem for ccache. Would some code be compiled more than once when compiling say, kdelibs ? |
If ccache can determine that a given compilation happend before (i.e. would result in the very same output as is already in the cache) the output in the cache is used and the compilation is not done a second time. But if you change code in the file, or change compile options, etc. the output of the new compilation would not be the same. Thus ccache cannot use the one in it s cache.
Cheers,
Andreas |
|
| Back to top |
|
|
Damasz
Tux's lil' helper
Joined: 24 Apr 2002
Posts: 91
Location: Groningen, the Netherlands
|
| Posted: Sat Jun 29, 2002 2:56 pm Post subject: |
|
|
Wouldn't it be cool if the cache could be shared on a p2p network between gentoo users? (and other linux users....)
You could download the files instead of compiling them if another user with the same settings as you allready compiled the program.
Then the biggest problem a lot of people have with gentoo (long compile time) would be gone! Sweet.....
Gert-Jan. |
|
| Back to top |
|
|
ph317
n00b
Joined: 02 Jun 2002
Posts: 43
|
| Posted: Wed Jul 03, 2002 6:02 pm Post subject: |
|
|
*edit* there was a long post here about ideas for a global network-wide compiler cache - after thinking over it for a weekend I decided it was a dumb idea for a whole host of reasons, and deleted the post so that nobody else wasted brain cycles reading it and thinking about it *edit*
Last edited by ph317 on Mon Jul 08, 2002 5:27 pm; edited 1 time in total |
|
| Back to top |
|
|
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
|
| Back to top |
|
|
tfoss
n00b
Joined: 04 Jul 2002
Posts: 8
|
| Posted: Fri Jul 05, 2002 8:29 pm Post subject: |
|
|
Just a warning:
I spent a couple hours trying to figure out why emerging kdelibs kept giving some odd libSTL... not found error. Turns out unmerging ccache fixed it.
-Ted |
|
| Back to top |
|
|
fghellar
Bodhisattva
Joined: 10 Apr 2002
Posts: 856
Location: Porto Alegre, BR
|
|
| Back to top |
|
|
micron
Guru
Joined: 23 Jul 2003
Posts: 411
Location: Bergamo, Italy
|
| Posted: Sat Sep 11, 2004 11:48 am Post subject: |
|
|
I've followed all the steps but
is empty... 
what can I do?
_________________
~ "Progress is merely a realisation of utopias" ~ |
|
| Back to top |
|
|
MooktaKiNG
Guru
Joined: 11 Nov 2002
Posts: 326
Location: London, UK
|
|
| Back to top |
|
|
micron
Guru
Joined: 23 Jul 2003
Posts: 411
Location: Bergamo, Italy
|
| Posted: Sun Sep 12, 2004 11:39 pm Post subject: |
|
|
Thanks a lot!
_________________
~ "Progress is merely a realisation of utopias" ~ |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
