Unsermake and KDE = Faster Compile?

caleb · Developer Joined: 02 Jun 2003 Posts: 404

Mod edit: Made this one a sticky - tomk
Unstuck on request by caleb --kallamej

Hi all.

I've added experimental unsermake support for kde builds. Using unsermake should cause long compile times for kde to come down significantly. Support is still very limited, and it's not guaranteed to work.

Here's how to make it happen:

1) Emerge kde-base/unsermake. This will install the unsermake scripts into /usr/kde/unsermake

2) Set your unsermake environment variable:
export UNSERMAKE="/usr/kde/unsermake/unsermake"

3) Try and emerge a kde program. Unsermake should be detected and will be used instead of the regular make system.

4) If for some reason you have a problem, just turn off unsermake support by "unset"ting the environment variable.

*It may take a few hours for the mirrors to grab the unsermake package, so be patient.

Please report successes and failures in this thread.

Caleb

Earthwings · Posted: Wed May 05, 2004 9:04 pm Post subject:

Sounds interesting. I did a quick google search because I never heard of unsermake before and found this page. Anyone knows what's the ugly name standing for?

If I got it right, the key idea is that recursive Makefiles are a bad thing as often they go through each dir to find out nothing has to be done or don't do everything and have to come back (and other problems). unsermake in turn uses one big Makefile. It's said it speeds up especially on SMP and distributed systems (distcc), but also on single CPU systems.

So I'll give it a try right now and tell if it works and if it's faster.

Edit: No, neither emerge sync brings it nor viewcvs.cgi on gentoo.org has it. Got to do it tomorrow...

Edit2: Nobody seems to notice this. You should give it a catchy title like "compile kde in half of the time" or similar

caleb · Developer Joined: 02 Jun 2003 Posts: 404

Oops - forgot the commit the ebuild to portage. Your bump reminded me, and it should be there now.

I've requested the thread to become sticky, and I've changed the title. Let's see if it takes off...

shm · Posted: Wed May 05, 2004 11:12 pm Post subject:

rfujimoto · Apprentice Joined: 22 Mar 2004 Posts: 195

Giving it a shot... would any app work? I was thinking `time emerge kaffeine` and then `export .... time emerge kaffeine`

Is that a good way to test it?

Edit:
Unfortunately I couldn't test it... something went wrong with the compile while using unsermake. A lot of these errors (replace [...] with a path)

norvillebarnes · Posted: Thu May 06, 2004 12:06 pm Post subject:

I gave this a spin last night with kdebase-3.2.2. It successfully compiled in just over an hour on my P4 1.8 256MB machine. I'm not sure how long it usually takes, but that seems like a fairly good improvment to me.
I especially like the supressed emerge output that came with this feature. kdebase compiling looked just like a 2.6 kernel compile, you could always tell exactly what was being done, and any warnings really stand out.
Thanks for a cool new tool caleb.

Al

AlterEgo · Veteran Joined: 25 Apr 2002 Posts: 1619

Earthwings · Posted: Thu May 06, 2004 1:06 pm Post subject:

@shm: Thanks for pointing this out. In fact I'm german but couldn't believe they meant the german word "unser"...

Tried unsermake in the meantime. No real improvement without SMP/distcc:

j-kidd · Apprentice Joined: 20 Feb 2003 Posts: 213

Successfully built kdeaccessibility, kdeaddons, kdeadmin, kdeartwork, kdeedu, kdegraphics, kdenetwork, kdepim, kdetoys, and kdeutils. Results of splat show definite reduction in build time (5% ~ 25%).

kdegames failed to compile with unsermake though.

Ian · Posted: Thu May 06, 2004 8:42 pm Post subject:

I don't know how much of a task this is, but how possible is it for make to be entirely replaced by unsermake?

I haven't tested it yet, but it sounds like this may be the start of some of the bigger projects getting 100% unsermake support (not right away of course, but as new versions come out). It'd be nice if everything could benefit from a 5 - 10% reduction in compiling time, because over a whole system, that's quite a bit of time.

I'm going to try this unsermake thing out when I get a chance, hopefully as soon as I'm done writing this

.

kevmille · Posted: Fri May 07, 2004 12:57 am Post subject:

No improvements here:

yngwin · Posted: Fri May 07, 2004 11:59 am Post subject:

Success here. But I don't know what to make of these times, I suspect different loads...

jstuart · n00b Joined: 03 Dec 2002 Posts: 57

Unsermake will REALLY shine if you're using distcc and make -j. If you're doing a single CPU compile.. well.. you won't get as much benifit from it as if you were doing a parallel compile.
_________________
Jeff Stuart
JAHGU (Just another happy Gentoo User)

asimon · Posted: Fri May 07, 2004 5:05 pm Post subject:

Ian · Posted: Fri May 07, 2004 7:47 pm Post subject:

shm · Posted: Sat May 08, 2004 4:39 am Post subject:

Has anyone tried unsermake with icecream yet?

(I suppose this would be most applicable to current distcc users)
_________________
what up

asimon · Posted: Sat May 08, 2004 8:44 am Post subject:

svyatogor · Posted: Sat May 08, 2004 3:37 pm Post subject:

Tried today updating kdelibs to 3.2.2. On my 2.6 GHz P4 with HyperThreading anf 1024 MB RAM it took it 60 minutes to compile, which is at least twice quicker than before.

Great job!

SpeedPhreek · Posted: Sun May 09, 2004 5:35 am Post subject:

I just tried unsermake with distcc and emerged kdelibs-3.2.2:

shm · Posted: Sun May 09, 2004 1:04 pm Post subject:

Turiya · n00b Joined: 12 Mar 2003 Posts: 24

Everything except arts (1.2.2) and kdegames (3.2.2) work fine with unsermake.
I didn't take times though.

int2str · Guru Joined: 29 Sep 2003 Posts: 362

arts 1.2.2 worked fine for me. But not kdegames-3.2.2. Though I dont even know if it compiles with automake yet.

Here's the error:

int2str · Guru Joined: 29 Sep 2003 Posts: 362

Update: Yes, kdegames 3.2.2 compiles fine with automake.
_________________
Adpot an unanswered post today!

Cottonee · Posted: Thu May 13, 2004 7:26 am Post subject:

teedog · Apprentice Joined: 09 Mar 2004 Posts: 211

Read about Unsermake here:
http://www.csh.rit.edu/slashdot/distcc.html

Sounds excellent. Hope it will work with other packages in the future.