Chromium is taking a long time to compile

[SoulDeaD]

[SoulDeaD]

[logrusx]

[logrusx]

I assembled a solution to build Chromium with individual cache, taking into account that it's being built with Clang now.
Using the following sources:
1) Current thread;
2) Optimizing ccache using per-package caches;
3) Knowledge Base:Overriding environment variables per package
4) Gentoo Project/Clang

I assembled the following solution:

/etc/portage/bashrc:

[Ant P.]

Posting in here just to warn everyone that chromium 65's (currently in ~arch) compile time is %$@£ing insane. It forces use of clang - which doesn't work with distcc and would invalidate ccache anyway if not for jumbo-build already breaking it. I saw it running a bundled copy of nodejs too despite $DEPENDing on the system nodejs package, there's probably more where that came from. The icing on the cake is that it spits unrecognised flag warnings for every compile command, because the build system adds a bunch of GCC-specific flags! Were they even thinking when they wrote this stuff?

This is still ongoing:

[mv]

[logrusx]

Regarding the compile time, no-jumboo build of v63 took nearly the same time as no-jumboo build of 65.
Regarding jumboo build and ccache, I think both are never going to work together. Even two consequent compilations of one and the same version were failing.
Regarding the warnings - they are harmless.
Regarding Firefox, I heard not very good things about Rust compilation speed, so you might not get better compile times with it. But it's worth a try.

[Ant P.]

[Zucca]

With such high compile times and not being able to use distcc and ccache, wouldn't it be more benefitical to use crossdev etc. and compile the whole mess on another PC althogether?
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

[logrusx]

Using distcc with clang is possible. It's just a matter of some research, modifications and some trial and error. There are quite a lot of resources on the Internet. Gentoo Wiki Clang page also confirms that:
Gentoo/Clang#distcc
Furthermore Clang is cross compiler and therefore it must be subject to distributed compiling:
Cross-compilation using Clang
Unfortunately with Google's "kill a fly with an atomic bomb" approach I think we can't do much about Chromium compile times. They use huge amount of cores and RAM, even on their personal working stations. With Jumboo build a full development Chromium build takes less than 20 minutes and they somehow recompile only the parts they've modified. How do you make people who have such resources on their disposal improve compilation process when they experience no necessity to do so? No way. Either Chromium takes a different direction, be it by forking a different project, or because of some fortunate circumstance forcing Google to do so, or we stay in reaction mode, ready to take whatever Google throws at us and deal with it. For now I don't see an option different than the later one, so I'm looking for ways to do my job. I prefer to be prepared and that's why I try this approach with non-jumboo build and ccache. If it works even partially I'll be fine.

[logrusx]

Test 1: Passed - completed a cached build.
It took a bit more than hour, most of which was python scripts execution. The uncached build took more than 30 hours, but I can't account for how much it was delayed because I did other, though not heavy stuff, maybe through 1/5 to 1/4 of the time.
The size of generated cache is 1.7G.
Now we wait for the next stable version, which is scheduled approximately for 12th of April, to see if there's any real benefit or if it even builds with cache.

[logrusx]

It appears that cache is barely useful, if of any use at all, for Chromium build.
Update from 65.0.3325.146 to 66.0.3359.117 is taking more than 14 hours already and is still going. Nothing suggest cache is even slightly useful.

[n05ph3r42]

check https://forums.gentoo.org/viewtopic-t-1074724.html

[logrusx]

[logrusx]

After all it turned out that individual cache for Chromium is not that useless.
Chromium 65.0.3325.146 took 100 000 seconds (~28 hours) to compile without jumboo build and like 1 hour to recompile from cache.
Chromium 66.0.3359.117 took twice as much, mainly because I was using the computer, which means the cache from 65 didn't do much, if anything to speed up the build.
Chromium 66.0.3359.139 however took only 12375 seconds (~3,5 hours) to recompile using the cache from 117, which is nearly 10% supposing that 66 would have taken nearly the same time as 65 if I hadn't used the computer.

Libreoffice seems to have benefited less than 10% decrease of the time between versions 6.0.1, 6.0.2, 6.0.3 and 6.0.4, noticing the decrease is greater between 6.0.3 and 6.0.4 and insignificant between 6.0.1 and 6.0.2.

The conclusion regarding Chromium is that although the build is quite slow for major versions, the benefit is significant between minor updates. The most important point for me is the system is usable while compiling without jumboo build as opposed to jumboo build with merge filke limit 10(that's how much my system can take), when at moments system almost freezes.

Given the above I find that experiment successful and the results useful. However people have way more powerful systems with more RAM these days and soon I'll get such as well, so for people with more modern systems cache might be almost irrelevant.

So people with old systems like mine (E6300 with 6G RAM) who need to use their systems while compiling might consider this option.

An important point is that I'm using ck-sources with Con Kolivas MUQSS scheduler to increase the system responsiveness.

[alinefr]

I wanted to build chromium as I don't want to use google-chrome binaries anymore, so I left my computer building while I went to work. I was expecting to be able to install the binary in the office as my home system serves binhost but I was afraid something was wrong because I could not find chromium in my binhost. Back home I was upset to find out that chromium is still building.

[logrusx]

[alinefr]

It finished!

[logrusx]

[weedy]

[mv]

[weedy]

[mv]

[LordArthas]

Hi!

Chromium just finished compiling on my box as well:

[nowhereman]

Building chromium also takes a huge amout of time for me. It seems to have started with version 66: