Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
using distcc and ccache to speed up compilations etc....
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
mikefot
Apprentice
Apprentice


Joined: 19 Nov 2014
Posts: 211

PostPosted: Tue Dec 05, 2017 11:05 pm    Post subject: using distcc and ccache to speed up compilations etc.... Reply with quote

Dear Folks,

Is it true that you could install e.g. distcc in Gentoo and then sign up on Amazon Cloud and recruit 10 cores to speed up compilations?

Has anyone tried this and succeeded or is it somewhat hyped?

Also how good is ccache in reality?

Regards

Michael Fothergill
Back to top
View user's profile Send private message
Etal
Veteran
Veteran


Joined: 15 Jul 2005
Posts: 1672

PostPosted: Sat Dec 09, 2017 2:44 am    Post subject: Reply with quote

Whether distcc will speed things up depends on how fast your internet is, how much slower your CPU is comparing to Amazon's, and what applications you're compiling.
  • Unless your Internet is vary fast, you''re likely spend a lot of time waiting for the source to upload/binary to download.
  • Ditto if you have a fast CPU – you could be compiling instead of waiting on network I/O.
  • You'll get the biggest speed-ups when compiling large C++ applications. Depending on the above, you may or may not benefit much with C applications since they're fast to compile. Small application won't benefit much since you'll be spending most of the time in configure phase and on linking, and the build itself won't parallelize much. Some applications don't build in parallel at all. Some code won't work well with distcc and it will fail remotely and then be re-compiled locally.
I have dual-core 800 MHz ARM device with Gigabit(ish) ethernet, and outsourcing the compilation to an AMD64 box with the crossdev ARM toolchain helps a lot. The ARM device can hardly keep up, so the load hardly even registers on the AMD64.

On the other hand, I tried once setting a number of similarly-specced AMD64s to use distcc but didn't notice much of a speedup, so I don't bother using it for that.

Also, you want to have the same compiler on your local and remote host, so you would want to run Gentoo on Amazon Cloud. And if you have Gentoo on the cloud, you might just as well build packages there entirely.

As for ccache, it can great if you're compiling the same code over and over again - a live ebuild that you like to update nightly, for instance. But, it will slow down the initial build and not all package builds will cache properly, so don't set it globally.
_________________
“And even in authoritarian countries, information networks are helping people discover new facts and making governments more accountable.”– Hillary Clinton, Jan. 21, 2010
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum