Per-package ccache-dir support in portage? (proposal)

[katabami]

Hi. I think per-package ccache-dir support in portage will be nice. Suppose, when a new option CCACHE_SPLIT is set to "yes", ccache is stored in <ccache_root>/<category>/<package>. It's good because:

* When the ccache repo is corrupt, you only have to delete the cache for that package.
* If portage prints the 'ccache -s' output before and after building, users can tune each cache size. Smaller cache is better, isn't it? (I don't know hash well, though.) From my experience, (initial use of cache) * 1.5 is a good size.

Personally, I don't use ccache much, because I don't do periodic "emerge -e world". On the other hand, I develop some packages, and I have a wrapper of emerge for these packages, like

[katabami]

I've got 2 new ideas, both of which depend on per-package ccache.

First, an option to use ccache only for rebuilding, or -r[0-9] upgrades. Otherwise, the cache is cleaned before emerging. Of course there'll be less cache hits. But ccache doesn't mean much between bumped versions.

For example, I turned on ccache for minimal upgrading of openssl, 1.0.0c -> 1.0.0d, and only 177/703 hits were there. And build time took longer with ccache. Other processes were running, so not accurate, though. From my experience, low hit rate, say 30%, doesn't improve the build time.

Upgrading of openssh 5.6_p1-r2 -> 5.8_p1 was a minimal bump in amd64 stable, and only 16/307 hits were seen.

My wild guesses (don't take too seriously) are:

• Smaller files tend to hit the cache more often.
  
• Compilation time is roughly proportional to the file size. OTOH cache seeking and reading overhead doesn't vary much, which is slower by orders. Remember that reading is blocking. So smaller files are far less efficient even with cache hit.

Cache access is slow. Deletion of ccache repository of 10M took me seconds on xfs. (I know ext4 is far better than xfs, especially in file deletion.)

So the second idea: Save the cache in tar file without compression. One tar for each package. When emerge begins, extract ccache files to tmpfs. When emerge is done, make a new tar, and replace the old one. So only one read and one write of HDD per package.

This trick may become soon obsolete in SSD era. I don't think that complicated procedure is good, but a good food for thought. :)

[Genone]

I think this is stuff for hooks, not for integration into the package manager (never been a fan of the distcc and ccache stuff in portage).