[SOLVED] problem using ccache........

[mikefot]

Dear Folks,

I installed ccache.

I think it is working because the no of files int he cache are increasing etc.

[mikefot]

Dear All,

I have been googling around and found some suggestions that I might need to change the permissions on the directories where the ccache files are stored.

Apparently the FEATURES beastie thinks the permissions are set a user level so it would be blocked entering directories set with root permissions.

Also the cache data can be stored in a root tree directory somewhere as well as in /var/tmp or wherever and some confusion can occur over which one is visited and used as well........

I will try some chmod chown and chgrp commands etc and see how I get on.

Regards

MF

[Irre]

When rebuilding same packages compile times are often much faster than first time (but not always).

With 8g RAM you can also allocate /var/tmp to tmpfs. In /etc/fstab:

tmpfs /var/tmp tmpfs size=4g,nosuid,nodev 0 0


BUT ccache must not be in tmpfs!

[mv]

[mikefot]

Many thanks for the responses.

If you look here:

https://forums.gentoo.org/viewtopic-t-1064518-start-0.html

then permissions and reading and writing of files do seem to be a bit of a vulnerability here.

But I think writing to the cache and probably reading from it to aid a second compilation IS actually occurring in my case now and maybe I don't need to play around with chmod, chown and chgrp etc............

This is also helpful:

http://wikigentoo.ksiezyc.pl/Ccache.htm

A suggestion of a package that ought to be speeded up a lot by ccache that I could test would be helpful.

Cheers

MF

[mikefot]

How about

[mikefot]

Dear All,

I will run with ccache for a while (2 months)

If I can't see any signs of serious speed up in compile times in a range of packages I will uninstall it and abandon it.

Cheers

MF

[Hu]

You only get a cache hit if ccache determines that reusing the cached file will produce the same result as compiling the input again. Some packages may have irrelevant but technically meaningful changes across rebuilds, which would prevent otherwise valid cache hits. For example, if the configure script writes a header that contains a string that varies with each run of configure, any source files that include that header will always miss the cache, even if in practice you don't care about that string being reused instead of rebuilt.

[mikefot]

Dear Folks,

A great many thanks for the comments provided so far.

I found a post that says that gcc is mostly written in C++ nowadays......

[mikefot]

Dear All,

I have started the first compilation run on gcc.

Here is the output of genelop on it:

[mikefot]

Dear All,

I have registered on the ccache mailing list.

I am going to see if anyone there can help with this.

Regards

MF

[mikefot]

Dear All,

The first run of gcc compilation has finished.

Here is the ccache output:

[mikefot]

Dear Folks:

I have started the second compilation of gcc again.

Here is the output of genlop after 5 minutes:

[krinn]

you should really know ccache is a dev tool more than an user tool, it will act good only if you rebuild something very similar (which imply cache hit), so for a gentoo user it mostly mean ccache will be of help for: a package that is revbump (going from -r1 to -r2), or a package that is rebuild for useflag change (however useflag change can lead to a way too much different code and cache miss).
it should also help a lot with autorebuild asked by portage, but in this case, it will in real mostly act bad ; if portage set r or R on the package it is because some other package has change and trigger the rebuild ; and in this case, you should not reuse cache hit, as you must not end with the same code finally ; if ccache hit and use old code, it will mostly void the r/R portage was forcing for a reason.

for gcc, it's because of its nature that ccache is doing bad: gcc is not building as-is, gcc build xgcc with sane default optimization and cflags, than xgcc will build the final gcc with your cflags and optimization set. While ccache may store the xgcc code for futher cache hit, when xgcc finally build gcc the code is than different (it now use your optimization) leading to cache miss ; and if ccache goes hit, again it will do wrong, as it will reinject code built for xgcc in your final gcc step (which remove xgcc optimization on gcc code).

if you know what you do and when/why ccache could lead to sneaky errors, maybe ccache could be really of help, but i would define this case only for devs and some users who handle it like it should (which doesn't mean just having portage handle it and thinking it will do good).
for most users, it mean strange compile errors and weird behaviour ; and if you have any trouble, first thing people will tell you: rebuild with ccache disable.
ccache is a good dev tool, but for user, a bad tool that when misuse will do more trouble than help.

most of the time (for a user), the case where you could use ccache with high efficiency with portage also mean you could had use binary of the package, and ccache couldn't beat binary emerge time.

i won't make friend again with ccache fans, but really the only sane rule with ccache should be: if it's to use it only with portage, it mean you should not use it.

[xaviermiller]

https://blog.flameeyes.eu/2010/07/debunking-ccache-myths-redux/
https://blog.flameeyes.eu/2008/06/debunking-ccache-myths/
_________________
Kind regards,
Xavier Miller

[mikefot]

Dear Folks,

I upgraded to ccache 3.3.4 from 3.2.4.....

I think there were some bugs in 3.2.4 that might have been a problem......

I also seem to have

[mikefot]

Dear Folks,

Ihave uninstalled distcc.

It was not set up any way.

Regards

MF

[mikefot]

Dear Folks,

I think ccache is starting to work a bit.

It is finishing compilations (if you sit through them) in a shorter time than genlop eta predicts on the second run.......

It does work.

I noticed on the ccache samba site that they mention something called cachecc1 which is meant to be a gcc cache ie it would speed up the compilation of gcc itself.

as I now know a gcc compilation takes 45 minutes on this kaveri box here.

If I would uninstall ccache here and manually install cachecc1 (using a make command or something - there is no gentoo package for cachecc1 AFAICT) I could compare it.

There is also a program called AO written by IBM I think that looks interesting but again there is no gentoo package for it.

Comments appreciated.

[mv]

[mikefot]

Dear All,

Many thanks for the tips and hints here.

What I am hearing hear is that ccache is actually OK for my purposes here relative to other compiler optimisers etc.

I am now compiling qtwebengine ETA 1 hr 41 minutes according to genlop for the first time around.

There is now 15 minutes to go left.

I will then run it again and see how I get on.

Regards

MF

[mikefot]

Dear All,

The compile time for qtwebengine was 2 hours 5 mins or so for the first run.

[mikefot]

Dear All,

It compiled qtwebengine the second time in 5 minutes flat............!

ccache works.

Regards

MF

[Hu]

[mikefot]

Dear All,

I emerged libreoffice under ccache and it took around 2hrs 2 mins on the first run.

I am now making the second compilation.

Regards

MF

[mikefot]

Dear All,

The second compilation has taken about 5 minutes or so again.

I am impressed.

MF