Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[TIP] Firefox and tmpfs: a surprising improvement
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
stevenrobertson
n00b
n00b


Joined: 28 Oct 2008
Posts: 8

PostPosted: Mon Jan 05, 2009 8:23 pm    Post subject: Reply with quote

ph030 wrote:
Nice idea, am modifying it for Opera.

One question though, what about squashing the profile, putting aufs/unionfs/... on top and bring all this in RAM, would that be beneficial? There are threads around to do it with portage, so it shouldn't be a problem to adopt it, just a mather of it making sense for this task or not.


It would work, but it wouldn't buy you much. IIRC aufs/unionfs/etc. do full-file copy-on-write, rather than differential storage, so the net effect would end up being almost identical in terms of RAM usage and speed boost (because stuff would get quickly copied to RAM as firefox writes to nearly everything in its profile directory).
Back to top
View user's profile Send private message
Ormaaj
Guru
Guru


Joined: 28 Jan 2008
Posts: 316

PostPosted: Mon Jan 05, 2009 8:38 pm    Post subject: Reply with quote

Putting /var on it's own partition might help. I use Polipo as a cache.

Last edited by Ormaaj on Thu May 17, 2012 2:57 am; edited 1 time in total
Back to top
View user's profile Send private message
xmaes
n00b
n00b


Joined: 13 Aug 2008
Posts: 69
Location: Madrid

PostPosted: Tue Jan 06, 2009 4:41 pm    Post subject: Reply with quote

thank you very much for the tips.
Went from nothing to firefox in tmpfs + polipo (cache in tmpfs) + privoxy + swichproxy ( from time to time i have problem downloading files from gmail when the proxy is enable).

i added .pack_ffox.sh to my .bash_profile and .bash_logout.
I added polipo -c /home/xavier/.polipo to my gnome session to setup polipo cache in my firefox profile because setting it the /etc/polipo/config file didnt work when running polipo wirth 'rc-update add'
I disable firefox's cache
i am not using gdm

I am not too sure if it is the best way to do it, i am still learning and experiencing...

it is fast and i am not getting those stupids adds anymore.

Xavier
Back to top
View user's profile Send private message
Enlight
Advocate
Advocate


Joined: 28 Oct 2004
Posts: 3517
Location: Alsace (France)

PostPosted: Thu Jan 15, 2009 7:43 pm    Post subject: Reply with quote

I'm sorry to ask, but I tried to mount my .mozilla on tmpfs and got _no_ mesurable improvement.

Did you guys who tried this after having FF to run did a echo 3 > /proc/sys/vm/drop_caches before running firefox again? Or did anybody saw and improvement on first launch?
_________________
le "lol" est aux boulets ce que le ";" est au programmeurs
Back to top
View user's profile Send private message
stevenrobertson
n00b
n00b


Joined: 28 Oct 2008
Posts: 8

PostPosted: Thu Jan 15, 2009 9:13 pm    Post subject: Reply with quote

Enlight wrote:
I'm sorry to ask, but I tried to mount my .mozilla on tmpfs and got _no_ mesurable improvement.

Did you guys who tried this after having FF to run did a echo 3 > /proc/sys/vm/drop_caches before running firefox again? Or did anybody saw and improvement on first launch?


I'm on an SSD currently, so this hack makes little measurable difference in startup speed. However, it continues to make a tremendous difference for anything that hits the databases - using the awesomebar (location bar), opening a tab, clicking on a link, etc. These activities are mostly write-bound, with the exception of reading autocomplete data, so the pagecache doesn't really matter.
Back to top
View user's profile Send private message
Enlight
Advocate
Advocate


Joined: 28 Oct 2004
Posts: 3517
Location: Alsace (France)

PostPosted: Thu Jan 15, 2009 9:54 pm    Post subject: Reply with quote

Wow I just trid to solve the problem statisticaly...

I ran strace -e open -o $file firefox, then put $file through a perl script to get my fields in dirname, basename, filedescriptor format.
I then submited that to oocalc datapilot asking him to count file accessed in each directory and splitting answers in fd < 0 or fd <= 0 cases.

Here are the results : at the time Firefox asks me wether i want my last session back or not, it has already opened 727 files! This number grows up to 1081 at the moment my pages are shown. Out of this 1081 files, 414 do not exists and out of this 414, 126 are in /usr/lib64/mozilla-firefox... WTF???!!!

still i have continued and came to that conclusion :

/usr/lib64/xulrunner-1.9 holds 252 files accessed at start time and weights 27Mb
/usr/lib64/mozilla-firefox holds 222 files accessed at start time and weights 4Mb
~/.mozilla/firefox/$blah.default holds 92 files accessed at start time and weights 72Mb

using tmpfs on theese 3 dirs would mean having 100Mb of ram holding 566 out of the 1081 files opened by FF at runtime (I.e. 52%)

so I did it, screwed the filecache with echo 3 > /proc/sys/vm/drop_caches and ran firefox

and it was... still slow as hell to launch :(

edit : seems the ENOENT problem in /usr/lib64/mozilla-firefox is related to the gnome support while not running gnome.

edit 2 : down to 820 files without gnome support.
_________________
le "lol" est aux boulets ce que le ";" est au programmeurs


Last edited by Enlight on Thu Jan 15, 2009 11:56 pm; edited 1 time in total
Back to top
View user's profile Send private message
SlashBeast
Moderator
Moderator


Joined: 23 May 2006
Posts: 2837

PostPosted: Thu Jan 15, 2009 10:31 pm    Post subject: Reply with quote

Enlight, you know how to do all of the cool stuff (how using strace etc. :P) so I have idea, can you find out why binary firefox from mozilla is lighter and imho faster than compiled from source?
Back to top
View user's profile Send private message
Sadako
Advocate
Advocate


Joined: 05 Aug 2004
Posts: 3753
Location: sleeping in the bathtub

PostPosted: Thu Jan 15, 2009 10:51 pm    Post subject: Reply with quote

SlashBeast wrote:
Enlight, you know how to do all of the cool stuff (how using strace etc. :P) so I have idea, can you find out why binary firefox from mozilla is lighter and imho faster than compiled from source?
Building www-client/mozilla-firefox with both custom-optimization and xulrunner USE flags disabled should actually result in a "faster", or at least more optimized firefox build, and one more similar to their binary builds too, so try that first.
_________________
"You have to invite me in"
Back to top
View user's profile Send private message
Palatis
n00b
n00b


Joined: 07 Oct 2006
Posts: 23
Location: Taipei/Taiwan

PostPosted: Sat Jan 17, 2009 1:38 am    Post subject: Reply with quote

I made a script to do it on a per-user basis, no need to touch "/etc/fstab".
it untars the tarball to an already-mounted tmpfs partition, in my case the "/tmp".
if your "/tmp" doesn't mount in tmpfs, you can always try the /dev/shm which is a glibc requirement.

WARNING: you have to manually tar the ~/.mozilla/firefox.tar file for the first time, or you destroy ALL your existing firefox profiles.
this can be done with
Code:
$ cd ~/.mozilla/
$ tar cf firefox.tar firefox

and here is the script...
Code:
#!/bin/bash

# change this if your /tmp is not a tmpfs.
TMPDIR="/tmp/.private/${USER}"
TMPARCHIVE="${TMPDIR}/firefox"
MOZDIR="${HOME}/.mozilla"
FXARCHIVE="${MOZDIR}/firefox.tar"

# initialize
if [ ! -d "${TMPARCHIVE}" ]
then
   mkdir -p "${TMPDIR}"
   tar xf "${FXARCHIVE}" -C "${TMPDIR}"
   ln -sfn "${TMPARCHIVE}" "${MOZDIR}/firefox"
fi

# run firefox
firefox

# finish up
if [ -e "${FXARCHIVE}" ]
then
   rm "${FXARCHIVE}"
fi
cd "${TMPDIR}"
tar -cf "${FXARCHIVE}" firefox
Back to top
View user's profile Send private message
teapot
Tux's lil' helper
Tux's lil' helper


Joined: 09 Nov 2006
Posts: 85
Location: Stockholm , Sweden

PostPosted: Fri Jan 23, 2009 12:25 am    Post subject: Reply with quote

Hopeless wrote:
SlashBeast wrote:
Enlight, you know how to do all of the cool stuff (how using strace etc. :P) so I have idea, can you find out why binary firefox from mozilla is lighter and imho faster than compiled from source?
Building www-client/mozilla-firefox with both custom-optimization and xulrunner USE flags disabled should actually result in a "faster", or at least more optimized firefox build, and one more similar to their binary builds too, so try that first.


Firefox and Openoffice can benefit a lot from using the -Os (optimize for size) compiler flag.
I moved the firefox user directory to RAM , following the instructions in this excellent guide!. I also recompiled firefox with the -Os flag. As long as I don't have an insane number of tabs , Firefox runs really smooth.
The performance glitches used to bug me , bot not anymore.
I don't know which flags the official builds use , but at least if you are using O3 you should notice quite a difference.
Some Gentoo users seem to believe that O3 is the way to go for a fast desktop.

I still have a growing anger within me over the sad fact that applications become more and more bloated with time though.
Firefox is an awesome browser by many means , but I do not believe that it is really necessary to access 800 (or what was it?) files during startup.
Back to top
View user's profile Send private message
Sadako
Advocate
Advocate


Joined: 05 Aug 2004
Posts: 3753
Location: sleeping in the bathtub

PostPosted: Fri Jan 23, 2009 1:09 am    Post subject: Reply with quote

teapot wrote:
Hopeless wrote:
SlashBeast wrote:
Enlight, you know how to do all of the cool stuff (how using strace etc. :P) so I have idea, can you find out why binary firefox from mozilla is lighter and imho faster than compiled from source?
Building www-client/mozilla-firefox with both custom-optimization and xulrunner USE flags disabled should actually result in a "faster", or at least more optimized firefox build, and one more similar to their binary builds too, so try that first.


Firefox and Openoffice can benefit a lot from using the -Os (optimize for size) compiler flag.
I moved the firefox user directory to RAM , following the instructions in this excellent guide!. I also recompiled firefox with the -Os flag. As long as I don't have an insane number of tabs , Firefox runs really smooth.
The performance glitches used to bug me , bot not anymore.
I don't know which flags the official builds use , but at least if you are using O3 you should notice quite a difference.
Some Gentoo users seem to believe that O3 is the way to go for a fast desktop.
Without "custom-optimization", the majority of firefox is actually compiled with -Os, and then other select parts/components are compiled with -O2.

I have pretty much everything compiled with -O2, but I'm starting to compile more and more with -Os via /etc/portage/env/.

When I started out with gentoo, I went with -O3 globally 'cause I didn't know any better, and tbh I'm surprised it didn't put me off gentoo altogether...

Quote:
I still have a growing anger within me over the sad fact that applications become more and more bloated with time though.
Same here, kinda makes me want to fetch a load of ebuilds from 2004 and see how well they still work...
_________________
"You have to invite me in"
Back to top
View user's profile Send private message
Ormaaj
Guru
Guru


Joined: 28 Jan 2008
Posts: 316

PostPosted: Tue Jan 27, 2009 6:54 am    Post subject: Reply with quote

Hopeless wrote:
I have pretty much everything compiled with -O2, but I'm starting to compile more and more with -Os via /etc/portage/env/.
Sorry for the O/T, but can you please explain or point me to some documentation on how to use / what format those files need to be in? All I could find about /usr/portage/env was a few vague forum references to it in a few blogs via google. IIRC man 5 portage doesn't even have that directory listed as having a function.
Back to top
View user's profile Send private message
Sadako
Advocate
Advocate


Joined: 05 Aug 2004
Posts: 3753
Location: sleeping in the bathtub

PostPosted: Tue Jan 27, 2009 8:31 am    Post subject: Reply with quote

Ormaaj wrote:
Hopeless wrote:
I have pretty much everything compiled with -O2, but I'm starting to compile more and more with -Os via /etc/portage/env/.
Sorry for the O/T, but can you please explain or point me to some documentation on how to use / what format those files need to be in? All I could find about /usr/portage/env was a few vague forum references to it in a few blogs via google. IIRC man 5 portage doesn't even have that directory listed as having a function.
Sure, tbh I can't even remember how/where I learned about it, but I know full well about the lack of documentation for this feature.

Take (for example) media-video/ffmpeg, if you have CFLAGS="-O2 -march=k8" in make.conf, but wish to have ffmpeg compiled with -O3, you'd create the file (and parent directories) '/etc/portage/env/media-video/ffmpeg', and simply add 'CFLAGS="-O3 -march=k8"' to it, basically the same format as make.conf, which is really just for setting shell variables.

That's basically all there is to it, and it has lots of uses, you can override many variables which you might set in make.conf, CFLAGS just being the most obvious one, but another fantastic one is EXTRA_ECONF, to which you can add extra --enable-foo or --disable-foo arguments to ./configure (presuming the ebuild uses econf).

One thing to bear in mind is that variables in these files override what you may find in make.conf, rather than add to them, so if you have just 'CFLAGS="-O3"' in the above example ffmpeg will be compiled without any -march set.

Another thing is not everything seems to work, one example of something which doesn't is ALSA_PCM_PLUGINS for media-libs/alsa-lib, and probably the same holds true for VIDEO_CARDS, LINGUAS and the like (run `emerge -v --info | grep USE_EXPAND` for a list), but most likely the reason for this is they are in fact just USE flags, and their real format is alsa_pcm_plugins_FOO, so adding alsa_pcm_plugins_rate and similar to /etc/portage/package.use should do the trick there.
_________________
"You have to invite me in"
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 8115

PostPosted: Wed Jan 28, 2009 2:13 am    Post subject: Reply with quote

Another option to query the state of USE_EXPAND would be portageq envvar USE_EXPAND, when checking the state of a single known variable it is somewhat faster.
Back to top
View user's profile Send private message
boundless
n00b
n00b


Joined: 06 Feb 2009
Posts: 1
Location: Tromsø, Norway

PostPosted: Fri Feb 06, 2009 10:09 am    Post subject: Re: [TIP] Firefox and tmpfs: a surprising improvement Reply with quote

stevenrobertson wrote:


<snip>

Changes:
2008-12-07 Added 'exec' to mount opts.
2008-12-05 Rewrote the guide based on feedback and a sore need for some editing.


I have applied this with my Firefox insttallation, but in Ubuntu and with FF 3.0.5 and when I try to update FF with ubuntuzilla.py it gives an error like this

    Backing up old Firefox preferences

    Traceback (most recent call last):
    File "/usr/local/bin/ubuntuzilla.py", line 1146, in <module>
    bs.start()
    File "/usr/local/bin/ubuntuzilla.py", line 209, in start
    fi.start()
    File "/usr/local/bin/ubuntuzilla.py", line 236, in start
    self.install()
    File "/usr/local/bin/ubuntuzilla.py", line 601, in install
    self.backupProfile()
    File "/usr/local/bin/ubuntuzilla.py", line 757, in backupProfile
    self.checkDiskSpaceForBackup("~/.mozilla")
    File "/usr/local/bin/ubuntuzilla.py", line 384, in checkDiskSpaceForBackup
    requiredForBackup = int(self.util.getSystemOutput(executionstring="du -sk "+target+" |awk '{print $1}'"))
    ValueError: invalid literal for int() with base 10: "du: cannot read directory `/home/kimme/.mozilla/firefox/koglgpp1.default/Cache': Permission denied"


How do I update FF with this tip enabled?
_________________
Kimme Utsi <http://kimme.blogspot.com/>
Back to top
View user's profile Send private message
sts
Tux's lil' helper
Tux's lil' helper


Joined: 02 Jul 2007
Posts: 97

PostPosted: Mon Feb 23, 2009 10:14 am    Post subject: Reply with quote

It might also speed things up noticeably if you vacuum the databases after firefox is closed:
Code:
$ for i in ~/.mozilla/firefox/*.*/*.sqlite; do sqlite3 $i vacuum; done
Back to top
View user's profile Send private message
nuhiNlow
n00b
n00b


Joined: 07 Jan 2009
Posts: 46

PostPosted: Wed Mar 04, 2009 11:36 pm    Post subject: Reply with quote

i put the .pack_ffox.sh in my user cron AND in gnome's startup programs..
how would i get it to execute upon shutdown of gnome?
Back to top
View user's profile Send private message
pdw_hu
Apprentice
Apprentice


Joined: 02 Jun 2008
Posts: 200
Location: Budapest, Hungary

PostPosted: Thu Mar 05, 2009 6:28 am    Post subject: Reply with quote

lysergia wrote:
i put the .pack_ffox.sh in my user cron AND in gnome's startup programs..
how would i get it to execute upon shutdown of gnome?


If by shutdown of gnome you mean turning off your computer then use /etc/conf.d/local.stop
Back to top
View user's profile Send private message
Lowspirit
Apprentice
Apprentice


Joined: 31 Jul 2002
Posts: 258
Location: Northern Sweden

PostPosted: Sun Mar 15, 2009 10:35 am    Post subject: Reply with quote

Tried this due to necessity because I have a raptor disk that's very loud and the disk reads during usage of Firefox would get ridiculous.

And I must say, succcess, startup is maybe only 1-2 seconds faster but usage overall I never hear my disk and certain elements that would clog it up before (using location bar etc) is just silky now, I use the method someone above mentions where I use my already mounted tmpfs that I have for my /tmp directory to just haul my profile to and then link my mozilla profile directory to that one, felt alot easier.
_________________
Gentoo | AMD X2 3800+ 2GB RAM | Kernel 2.6.30 . ReiserFS . CFQ . GCC4.3.3 | Firefox 3.5 | Gnome 2.26 w/ Compiz-Fusion
"Penguins are the only fish that can fly"
Back to top
View user's profile Send private message
Dont Panic
Guru
Guru


Joined: 20 Jun 2007
Posts: 320
Location: SouthEast U.S.A.

PostPosted: Mon Mar 16, 2009 10:27 pm    Post subject: Reply with quote

pdw_hu wrote:
lysergia wrote:
i put the .pack_ffox.sh in my user cron AND in gnome's startup programs..
how would i get it to execute upon shutdown of gnome?


If by shutdown of gnome you mean turning off your computer then use /etc/conf.d/local.stop


Since /etc/conf.d/local.stop runs as root, is there a generic way to put this in /etc/conf.d/local.stop, and have it work for my normal user (that doesn't involve hard-coding the normal user)?
Back to top
View user's profile Send private message
razze
Apprentice
Apprentice


Joined: 09 Mar 2005
Posts: 159
Location: Espoo, Finland

PostPosted: Tue Mar 17, 2009 7:36 pm    Post subject: Reply with quote

Thanks!

This was a great tip! On my rig (amd X2 3800+, 2G mem) the difference is clearly noticeable and it feels like a new computer.
_________________
--------------------------
Linux User #393524

AMD Athlon 64 X2 3800+ @2,3 GHz, Asus A8V, VIA K8T800PRO, Nvidia GeForce 7600GS, 2 Gb Dual Channel DDR RAM, 2x Samsung SP120 200Gb, 2x WD 320Gb, Plextor PX-716A, linux-2.6.31-gentoo-r10
Back to top
View user's profile Send private message
shgadwa
Guru
Guru


Joined: 12 Mar 2009
Posts: 326

PostPosted: Wed Mar 18, 2009 8:40 am    Post subject: Reply with quote

I get stuck here....

Quote:
localhost firefox # $ "${HOME}/.pack_ffox.sh"
bash: $: command not found
localhost firefox # ${HOME}/.pack_ffox.sh
bash: /root/.pack_ffox.sh: No such file or directory
localhost firefox # ${HOME}/.pack_ffox.sh
bash: /root/.pack_ffox.sh: No such file or directory
localhost firefox # ${HOME}/shawn.pack_ffox.sh
bash: /root/shawn.pack_ffox.sh: No such file or directory
localhost firefox # ${HOME}/.pack_ffox.sh
bash: /root/.pack_ffox.sh: No such file or directory
localhost firefox # {HOME}/.pack_ffox.sh
bash: {HOME}/.pack_ffox.sh: No such file or directory


I must have done something wrong. I thought I was supposed to create a ${HOME}/.pack_ffox.sh file.... right?
Back to top
View user's profile Send private message
shgadwa
Guru
Guru


Joined: 12 Mar 2009
Posts: 326

PostPosted: Wed Mar 18, 2009 9:21 am    Post subject: Reply with quote

I do not get it. The thing is, I do not even have a profile directory.... I have a file called profiles.ini.

I realize I'm supposed 'run the script' not save the file.... I think. But I cannot run the script too easily either.


Quote:
localhost firefox # #!/bin/bash
localhost firefox #
localhost firefox # # Change this to match your correct profile
localhost firefox # PROFILE="profile.ini"
localhost firefox #
localhost firefox # cd "${HOME}/.mozilla/firefox"
localhost firefox #
localhost firefox # if test -z "$(mount | grep -F "${HOME}/.mozilla/firefox/${PROFILE}" )"
> then
> mount "${HOME}/.mozilla/firefox/${PROFILE}"
> fi
[mntent]: line 20 in /etc/fstab is bad
mount: can't find /root/.mozilla/firefox/profile.ini in /etc/fstab or /etc/mtab
localhost firefox #
localhost firefox # if test -f "${PROFILE}/.unpacked"
> then
> tar --exclude '.unpacked' -cpf packed.tmp.tar "$PROFILE"
> mv packed.tar packed.tar.old
> mv packed.tmp.tar packed.tar
> else
> tar xpf packed.tar &&\
> touch "${PROFILE}/.unpacked"
> fi
bash: : command not found
touch: cannot touch `profile.ini/.unpacked': No such file or directory
localhost firefox # #!/bin/bash
localhost firefox #
localhost firefox # # Change this to match your correct profile
localhost firefox # PROFILE="profile.ini"
localhost firefox #
localhost firefox # cd "${HOME}/.mozilla/firefox"
localhost firefox #
localhost firefox # if test -z "$(mount | grep -F "${HOME}/.mozilla/firefox/${profile.ini}" )"
> then
> mount "${HOME}/.mozilla/firefox/${profile.ini}"
> fi
bash: ${HOME}/.mozilla/firefox/${profile.ini}: bad substitution
bash: ${HOME}/.mozilla/firefox/${profile.ini}: bad substitution
localhost firefox #
localhost firefox # if test -f "${profile.ini}/.unpacked"
> then
> tar --exclude '.unpacked' -cpf packed.tmp.tar "$PROFILE"
> mv packed.tar packed.tar.old
> mv packed.tmp.tar packed.tar
> else
> tar xpf packed.tar &&\
> touch "${profile.ini}/.unpacked"
> fi
bash: ${profile.ini}/.unpacked: bad substitution
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Wed Mar 18, 2009 9:56 am    Post subject: Reply with quote

belikeyeshua wrote:
...


ls -l ~/.mozilla/firefox :?:
_________________
The End of the Internet!
Back to top
View user's profile Send private message
shgadwa
Guru
Guru


Joined: 12 Mar 2009
Posts: 326

PostPosted: Wed Mar 18, 2009 10:27 am    Post subject: Reply with quote

Quote:
lrwxrwxrwx 1 root root 26 Mar 18 09:13 firefox -> /tmp/.private/root/firefox
drwx------ 6 root root 4096 Mar 18 09:13 hzotbwlp.default
-rw-r--r-- 1 root root 10240 Mar 18 08:24 packed.tar
-rw-r--r-- 1 root root 94 Mar 16 08:31 profiles.ini
localhost shawn #


I dunno... maybe its working now, or maybe I have to reboot or something or the other.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4  Next
Page 2 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum