Where and How? Firefox Compile Optimization.

Message

HecHacker1 · Post by **HecHacker1** » Sat Apr 01, 2006 12:08 am

So I visit these forums from time to time:

http://forums.mozillazine.org/viewforum.php?f=42

and they have custom compiled builds of firefox for different archs. However, none match my particular platform and I want to know where and how I can edit the ebuild/mozconfig that builds firefox under Gentoo? I use ~x86.

Basically, I want to be able to do an "emerge mozilla-firefox" but with my own compile options, like -Os instead of the forced default of -O2. I want to try and speed up the loading of firefox because it just takes so long on linux. I know that the Gentoo dev's forced those defaults in the spirit of squashing/preventing bugs, but I am willing to deal with it and I won't complain/file errornous bugs.

anybody care to point me in the right direction? Thanx!

cynric · Post by **cynric** » Sat Apr 01, 2006 1:41 am

I believe this is just a matter of playing with the ebuilds and the inherited eclasses. I remember looking for this myself, but didn't stick with it long enough to finally get -Os to work although I did find it in the mozilla eclass. However, I think a new eclass was released since I last tried. Would be interested in hearing the results of this.

Suicidal · Post by **Suicidal** » Sat Apr 01, 2006 2:06 am

Have you tried prelinking? I doubt you will get much improvement from -O2 to -Os.

cynric · Post by **cynric** » Sat Apr 01, 2006 2:32 am

Isn't prelinking only possible in KDE? Setting LDFLAGs would work though. Be aware that setting LDFLAGs can cause problems when bugging stuff. Make sure before you file one that you've recompiled the application without LDFLAGs to rule them out.

HecHacker1 · Post by **HecHacker1** » Sat Apr 01, 2006 2:44 am

yes, i use Prelinking and LDFlag options, and while this did provide a significant overall boost to my system responsiveness, Firefox/THunderbird take a long time to start from a cold boot. And yes, KDE benefits the most from prelinking, I didn't really notice a speed increase on my gnome desktop (my main desktop).

Luckily I do have a lot of ram, so most of my applications are cached.. but I also tend to restart my system a lot (working on getting Modular X/XGL with my laptop, hence it frequently crashes when i use compiz).

And compiler optimizations on firefox do make a significant difference.. or can. On my windows partition I use one of the optimized firefox builds and it is way faster than normal firefox (in startup and page rendering). Not to mention there are a lot of patches for firefox to improve performance on the code level.

So does anybody want to tell me where to begin? you mentioned eclass files, where are they? I could do a systemwide root search... but why hunt and guess.

cynric · Post by **cynric** » Sat Apr 01, 2006 3:00 am

True, I should have said said "mostly" and not "only" wrt to KDE and prelinking.

Popping open the mozilla-firefox-1.5.0-r3 ebuild shows that the following are inherited:

Code: Select all

flag-o-matic toolchain-funcs eutils mozconfig-2 mozilla-launcher makeedit multilib fdo-mime mozextension autotools

These are located in /var/portage/eclass/ (or wherever you have $PORTDIR). It looks like mozconfig is what you want. Although it's not listed in the ebuild, it is inherited from within mozconfig-2. Hope that helps.

HecHacker1 · Post by **HecHacker1** » Sat Apr 01, 2006 4:11 am

thanx.. currently building with -Os. I had to change mozcoreconfig before any effect was noticed. I'm guessing somewhere that is inherited.

Now I need to figure out a way to turn off disk usage when using firefox... you see, what I want to do is make firefox run completely in memory. That way my laptop hard drive can turn off while I use firefox.

By default firefox stores files onto the disk for cache, but unfortunately that keeps my hard drive running just for browsing the web, and hence making lots of noise and using more power.

I can turn off the disk cache in about:config, or set it to 0, but then performance of the back/forward suffers because firefox doesn't want to retain all the images in memory, and has to constantly download them.

I can't understand why firefox doesn't store pages in memory.. it seems to only store a limited amount even though I set a rediculously high limit (300MB for memory cache, I have 1.2GB of RAM in my laptop just for this purpose).

If I can't configure firefox to store pages in memory, then I may have to use a RAM disk and tell it to store it's browser disk cache there.. thereby putting it all in memory.

what do you think? If I could somehow get this working I would like to share the results. A "laptop friendly" firefox that runs completely in ram, thus providing faster performance without the need for the hard drive.

cynric · Post by **cynric** » Sat Apr 01, 2006 4:23 am

I don't know of any feature in firefox that does that. There could be one of course, I simply don't know as I haven't had the need, but I see your point. Going with a ramdisk is the only solution I know of. I'm interested in what kind of perfomance -Os yields so please keep us informed.

HecHacker1 · Post by **HecHacker1** » Sat Apr 01, 2006 5:45 am

FYI, I finished compiling firefox with -Os and other tweaks and I am posting from it right now

but I still am compiling kdelibs so it might be a while before I get to try a cold boot and see how well firefox starts up for the first time.

but so far it appears as if everything in firefox is working as usual. It definately feels faster. Especially when I open a new firefox window with an already open firefox instance. It opens instantaneously when I click. Before there was a slight lag.

I also gave up on trying to get firefox to try and store everything in memory. The only solution is to use tmpfs and mount it at boot using my /etc/fstab:

Code: Select all

none	/home/hechacker1/.mozilla/firefox/33pms5do.default/Cache tmpfs	rw,user,uid=1000

just point it to where firefox stores your cache (check using about:cache) and set the correct user id for your user. Now my hard drive can finally sleep. And page loading is tremendously faster. will keep you updated

cynric · Post by **cynric** » Sat Apr 01, 2006 5:47 am

Very nice. Thanks for the rundown and glad you got it up.

KD-120RD · Post by **KD-120RD** » Sat Apr 01, 2006 2:06 pm

Hello I find firefox quite slow on linux, especially when compared to konqueror.

So this sounds interesting. I'm compiling firefox with -Os right now. Lets see what it gives.

How about compiling with gtk+1.2? Damn Small Linux does it, so if it works will hopefully reduce startuptime as well.

Adrian

KD-120RD · Post by **KD-120RD** » Sat Apr 01, 2006 3:03 pm

Recompiled with -Os, but the size is the same as before, but I'm quite sure I did things right. I hope ...

I added

Code: Select all

elif is-flag -Os; then
                mozconfig_annotate "from CFLAGS" --enable-optimize=-Os

to mozconfig.eclass in the "CFLAGS setup and ARCH support" part.

emerge output:

Code: Select all

==========================================================
Building mozilla-firefox-1.0.7-r4 with the following configuration
...
--enable-optimize=-Os           from CFLAGS

Update on gtk1 compilation:
Seems like forcing gtk1 brakes things:

Code: Select all

i686-pc-linux-gnu-g++ -o nsImageGTK.o -c -DNATIVE_THEME_SUPPORT -DOSTYPE=\"Linux2.6\" -DOSARCH=\"Linux\" -DUSE_POSTSCRIPT -I../.. -I./. -I./.. -I./../shared -I./../freetype -I./../x11shared -I/usr/include/freetype2 -I/usr/include/freetype2/config    -I../../../dist/include/xpcom -I../../../dist/include/string -I../../../dist/include/widget -I../../../dist/include/view -I../../../dist/include/util -I../../../dist/include/pref -I../../../dist/include/uconv -I../../../dist/include/unicharutil -I../../../dist/include/locale -I../../../dist/include/necko -I../../../dist/include/content -I../../../dist/include/dom -I../../../dist/include/layout -I../../../dist/include/imglib2 -I../../../dist/include/gfx -I../../../dist/include -I/var/tmp/portage/mozilla-firefox-1.0.7-r4/work/mozilla/dist/include/nspr        -fPIC  -DGENTOO_NSPLUGINS_DIR=\"/usr/lib/nsplugins\" -DGENTOO_NSBROWSER_PLUGINS_DIR=\"/usr/lib/nsbrowser/plugins\"  -fno-rtti -fno-handle-exceptions  -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -march=athlon-xp -pipe -Wno-deprecated -Wno-return-type -w -fshort-wchar -pthread -pipe  -DNDEBUG -DTRIMMED -ffunction-sections -Os -I/usr/include/gtk-1.2 -I/usr/include/glib-1.2 -I/usr/lib/glib/include   -DGENTOO_NSPLUGINS_DIR=\"/usr/lib/nsplugins\" -DGENTOO_NSBROWSER_PLUGINS_DIR=\"/usr/lib/nsbrowser/plugins\"  -DMOZILLA_CLIENT -include ../../../mozilla-config.h -Wp,-MD,.deps/nsImageGTK.pp nsImageGTK.cpp
In file included from nsGfxFactoryGTK.cpp:66:
nsFontMetricsPango.h:47:25: pango/pango.h: No such file or directory
In file included from nsGfxFactoryGTK.cpp:66:
nsFontMetricsPango.h:235: error: ISO C++ forbids declaration of `PangoFontDescription' with no type
nsFontMetricsPango.h:235: error: expected `;' before '*' token
nsFontMetricsPango.h:236: error: ISO C++ forbids declaration of `PangoContext' with no type
nsFontMetricsPango.h:236: error: expected `;' before '*' token
nsFontMetricsPango.h:237: error: ISO C++ forbids declaration of `PangoContext' with no type
nsFontMetricsPango.h:237: error: expected `;' before '*' token
nsFontMetricsPango.h:238: error: ISO C++ forbids declaration of `PangoContext' with no type
nsFontMetricsPango.h:238: error: expected `;' before '*' token
nsFontMetricsPango.h:239: error: ISO C++ forbids declaration of `PangoAttrList' with no type
nsFontMetricsPango.h:239: error: expected `;' before '*' token
nsFontMetricsPango.h:274: error: `PangoLayoutLine' has not been declared
nsFontMetricsPango.h:275: error: ISO C++ forbids declaration of `aLine' with no type
nsFontMetricsPango.h:287: error: `PangoLayout' has not been declared
nsFontMetricsPango.h:287: error: ISO C++ forbids declaration of `aLayout' with no type
gmake[4]: *** [nsGfxFactoryGTK.o] Error 1
gmake[4]: *** Waiting for unfinished jobs....
gmake[4]: Leaving directory `/var/tmp/portage/mozilla-firefox-1.0.7-r4/work/mozilla/gfx/src/gtk'

Maybe I'll find somehting about this in the forums.

yngwin · Post by **yngwin** » Sat Apr 01, 2006 3:37 pm

GTK1 is no longer supported by Mozilla, so if you want that, you're on your own (or try Skipstone).

cynric · Post by **cynric** » Sat Apr 01, 2006 5:31 pm

I believe that if you want to use -Os that you'll need to comment out your CFLAGs if you have any set. You are probably setting -Os but other customizations are still being included thus negating your effects. Try something like:

Code: Select all

CFLAGS="-march=whatever -Os -pipe"

FireBurn · Post by **FireBurn** » Sat Apr 01, 2006 11:02 pm

By default firefox stores files onto the disk for cache, but unfortunately that keeps my hard drive running just for browsing the web, and hence making lots of noise and using more power.

Have you thought about using a ram disk and pointing your disk cashe to it?

Mike

cynric · Post by **cynric** » Sun Apr 02, 2006 2:08 am

HecHacker did:

HecHacker1 wrote:I also gave up on trying to get firefox to try and store everything in memory. The only solution is to use tmpfs and mount it at boot using my /etc/fstab:
Code: Select all
none	/home/hechacker1/.mozilla/firefox/33pms5do.default/Cache tmpfs	rw,user,uid=1000
just point it to where firefox stores your cache (check using about:cache) and set the correct user id for your user. Now my hard drive can finally sleep. And page loading is tremendously faster. will keep you updated

nxsty · Post by **nxsty** » Sun Apr 02, 2006 7:08 am

@KD-120RD

Reemerge pango and if it still doesn't work reemerge gtk also. Your problem doesn't seem to be related to CFLAGS.

KD-120RD · Post by **KD-120RD** » Sun Apr 02, 2006 11:44 am

@cynric: I did just that. Maybe, by default, the the strict restrictions of the mozilla devs don't allow any cflags that blow your binary?

@HecHacker1: How much smaller did you get your /usr/bin/firefox and what cflags did you use before?

@nxsty: will try that later.

ruben · Post by **ruben** » Sun Apr 02, 2006 12:28 pm

HecHacker1 wrote:I also gave up on trying to get firefox to try and store everything in memory. The only solution is to use tmpfs and mount it at boot using my /etc/fstab[...]
Now my hard drive can finally sleep. And page loading is tremendously faster.[...]

Great tip. I'm having the same problem with firefox (even though I use laptop-mode-tools to get the disk to sleep anyways), this is definitely something I'm gonna do. My idea is to just copy the whole profile into a tmpfs drive at boot and copy it back at shutdown. Hmm.. might actually use a tgz-file for that.

KD-120RD · Post by **KD-120RD** » Sun Apr 02, 2006 1:13 pm

Remerging pango and gtk+ didn't help. mozilla-firefox fails with the same errors again.
I guess now that gtk+ is not longer supported, it will not be that easy to get it running.

I'm using -truetype for mozilla-firefox now. That helps a little.

cynric · Post by **cynric** » Sun Apr 02, 2006 5:01 pm

KD-120RD wrote:@cynric: I did just that. Maybe, by default, the the strict restrictions of the mozilla devs don't allow any cflags that blow your binary?

I don't think it strips all CFLAGs, but I'd have to check. It will definately strip out known, problematic CFLAGs.

HecHacker1 · Post by **HecHacker1** » Sun Apr 02, 2006 7:13 pm

Yeah.. my cflags are simple. -Os -fomit-frame-pointer -pipe.

In the mozconfig file I commented out the "strip-flags" option and just made the "mozilla-fallback" option to use -Os since it defaults to that anyways.

I finally got around to cold booting my newly recompiled -Os system.

1. The ram usage is MUCH lower than it was before, for my whole system overall. Now I idle around 240MB of usage with full X and effects going. Before it was like 320.

2. Everything starts up faster and with less disk usage than -O2.

3. Firefox starts up slightly faster than before.. it's only a subjective test because I didn't actually time it, but firefox starts in about 3 seconds now from cold boot. I know that the binary's are not really that much smaller, but when the many parts of the system are recompiled with -Os it helps.

4. I still haven't figured out how to completely eliminate disk usage by firefox. I suspect I may have to copy my entire profile into memory because firefox still spins the disk up from time to time.

5. Personally I like the newer gtk because it blends so well with my gnome environment. I also saw that firefox has support for qt too.

I think this thread is pretty good. Maybe i'll start a wiki on optimizing firefox/thunderbird and see what people contribute.

cynric · Post by **cynric** » Sun Apr 02, 2006 7:22 pm

I have fairly lengthy CFLAGs which cut down on compile time. But, with the ck-sources, my memory and swap usage have gone up. Since my machine is amd64 with 1G of RAM, compile times aren't doing it for me any more. I've heard of many positive results from -Os systems so I've been tempted to go that direction and see what kind of effects that has.

I take that firefox was the only ebuild you modified and that by "newly recompiled -Os system" you mean just the CFLAG option in make.conf? Thanks for keeping us updated.

HecHacker1 · Post by **HecHacker1** » Mon Apr 03, 2006 4:08 am

yeah. i set my cflags to -Os -pipe -fomit-frame-pointer in make.conf and then did emerge -e gcc to try and recompile the important parts of the system. Then I recompiled my gnome desktop.

cynric · Post by **cynric** » Mon Apr 03, 2006 4:37 am

Cool. Thanks. Upgrading to gcc-4.1 from 4.0 right now to see if that fixes some compile problems. After I ensure that everything still works, I plan to try -Os myself -- something I've been meaning to try for a while now.