View previous topic :: View next topic |
Author |
Message |
dennis Tux's lil' helper
Joined: 06 Jun 2002 Posts: 84
|
Posted: Sat Oct 12, 2002 2:22 am Post subject: RAM disks and Portage = more speed ? |
|
|
Hi,
I guess you all know that slow Harddisks, can affect the execution-speed of some programms really bad To 'save acces times' the UT-Live CD can 'cache' itself in the computers memory. Now i thought that I could as well speed up portage by loading its temp directories in a RAM disk ... what directories does portage compile in? would this be worth it?
dennis schaaf |
|
Back to top |
|
|
rac Bodhisattva
Joined: 30 May 2002 Posts: 6553 Location: Japanifornia
|
Posted: Sat Oct 12, 2002 3:19 am Post subject: Re: RAM disks and Portage = more speed ? |
|
|
dennis wrote: | what directories does portage compile in? |
/var/tmp/portage by default. Configurable by setting PORTAGE_TMPDIR.
Quote: | would this be worth it? |
IMO, only if it were able to have several scratch directories, and fail over seamlessly when one filled up. Otherwise, when compiling big packages like X you had better have a boatload of RAM. _________________ For every higher wall, there is a taller ladder |
|
Back to top |
|
|
fghellar Bodhisattva
Joined: 10 Apr 2002 Posts: 856 Location: Porto Alegre, BR
|
|
Back to top |
|
|
dennis Tux's lil' helper
Joined: 06 Jun 2002 Posts: 84
|
Posted: Sat Oct 12, 2002 3:58 pm Post subject: makes sence |
|
|
hi
yea i can immagine that it requires a lot of space ...
but would it be possible to create such a fail over system?
[random idea]maybe you could software raid the RAM disk and a harddisk but that seems to be a more complicated than neccessary method for me ...[/random idea]
dennis |
|
Back to top |
|
|
phypor n00b
Joined: 25 Jun 2002 Posts: 68 Location: Texas
|
Posted: Sun Oct 13, 2002 2:08 am Post subject: RAMdisk vs kernel caching |
|
|
dennis wrote: | hi
yea i can immagine that it requires a lot of space ...
but would it be possible to create such a fail over system?
[random idea]maybe you could software raid the RAM disk and a harddisk but that seems to be a more complicated than neccessary method for me ...[/random idea]
dennis |
The OS cacheing will (very smartly) handle file caching for you...
The more free ram you have, the more it will grab for file cache, which will be released as need for application memory.
run free or top in another console when you emerge, youll see the 'cached' bouncing around as files are cached and purged.
ram disks for file access intensive stuff are not the best way...
let the kernel handle your excess ram with dynamic caches.
(until you get over a certain amt of ram (1 gig? 4?) which will be used automatically as ram disk for swapping only)
~phypor _________________ ALWAYS stay away from tanks enguled in fire.
- 2000 Emergency Response Guidebook, pg307 |
|
Back to top |
|
|
dreamer3 Guru
Joined: 24 Sep 2002 Posts: 553
|
Posted: Fri Nov 15, 2002 11:39 am Post subject: Re: makes sence |
|
|
dennis wrote: | yea i can immagine that it requires a lot of space ...
but would it be possible to create such a fail over system?
[random idea]maybe you could software raid the RAM disk and a harddisk but that seems to be a more complicated than neccessary method for me ...[/random idea] |
What you want is already done for you with the tmpfs filesystem. It is a file system that resides in ram but can be paged to disk as the OS deams necessary... Just make sure you have a LARGE enough swap space for compiles (over 2 gig for Open Office) and
Code: |
mount tmpfs /var/tmp/portage -t tmpfs -o size=1250m
|
You will need tmpfs compiled into your kernel for this to work... By default it only uses half the available memory though so you'd want to use the size paramter to set it to RAM + SWAP - some sane amount so it wouldn't be possible to use 100% of storage for tmpfs...
When I get around to repartitioning my system (again) I think I might set it up this way... I have 768mb of RAM. I wouldn't recommend this for < 512mb...
EDIT: OK, I just created a 1.25gig tmpfs mounted at /var/tmp/portage (I have 1.5 gig including swap). Now to find something to compile |
|
Back to top |
|
|
dennis Tux's lil' helper
Joined: 06 Jun 2002 Posts: 84
|
Posted: Fri Nov 15, 2002 11:31 pm Post subject: |
|
|
can you compile whatever you plan to compile twice? once with tmpfs and once without?
dennis |
|
Back to top |
|
|
dreamer3 Guru
Joined: 24 Sep 2002 Posts: 553
|
Posted: Sat Nov 16, 2002 12:44 am Post subject: |
|
|
Hehehe, for a real speed comparison, eh? I imagine it's more subjective than actual performance... The only real speed increase would come from less disk I/O, and I have a decently fast disk... for a slow HD I think there would be a bigger difference seen.
A benchmark also wouldn't work because I'm always using my PC for other tasks in the foreground and of course that would effect things. Sorry.
I compiled a bunch of small games and stuff (DOOM, yeah!). They went fast (cause they were small) but the thing I loved was no disk access... whisper quiet... me like. I'm going to leave it like this for a while...
Just try it, not like it'll hurt anything... you can always undo it with the press of the enter key.
Code: | mount tmpfs /var/tmp/portage -t tmpfs -o size=1024m |
to undo it
Code: | umount /var/tmp/portage |
It should go without saying don't change this in the MIDDLE of emerging something... You will need to change the size parameter to suit your system... I'm using 1250m, but only because I can't repartition now to get a bigger swap file... If I could I'd probably set it for 2.5 gig and a 3 or 4 gig swap...
When doing it permentantly I'd probably mount it on /tmp, then create a /tmp/var -> /var/tmp symlink, so all my temp files are on the tmpfs.
Edit: You could also use the ramit script (above) to manage this for you, though you (I will when I get around to it) would have to be using the tmpfs mode and add the size parameter to the mount command.
Code: | ramit /tmp
ramit /var/tmp |
If you currently have a lot of junk in your /tmp or /var/tmp files it would do to flush them out before using ramit since it copies the files over before mounting tmpfs. |
|
Back to top |
|
|
TheEternalVortex Apprentice
Joined: 15 Oct 2002 Posts: 207 Location: San Jose, CA
|
Posted: Sat Nov 16, 2002 2:28 am Post subject: |
|
|
I was interested in this, so I made a quick (unscientific) test:
armagetron-0.1.4.9
emerge on regular hard drive: 3m 27s
emerge on 128 MB ramdisk: 2m 18s
I would like to point out that my hard drive is very slow (15 MB/s disk reads by hdparm). Still, it seems to provide a nice benefit. _________________ -- Andy |
|
Back to top |
|
|
tkdack n00b
Joined: 12 Apr 2002 Posts: 44 Location: Sydney, Australia
|
Posted: Sat Nov 16, 2002 5:38 am Post subject: |
|
|
My box has ~780MB of RAM and the same of swap.
By default I have a tmpfs mount of 768M set up for /var/tmp
Using this and a 2GB ccache my compiles are considerably faster (KDE version and release bumps in around 2 hours on a 1GHz Athlon)
For the larger packages that need heaps of swap (OOo and j2sdk) I have a spare partition that I mount for the purpose and I take the performance hit.
So far those are the only two apps that I need to use a physical disk for, X will just fit into the 768M tmps file system.
During most normal builds my system is running KDE in X with a heap of open applications, including a VMWare session and the box rarely uses more than 10% of swap, if that much. _________________ Troy
http://linux.tkdack.com
Former Gentoo contributor TaD |
|
Back to top |
|
|
dreamer3 Guru
Joined: 24 Sep 2002 Posts: 553
|
Posted: Sat Nov 16, 2002 5:59 am Post subject: |
|
|
tkdack wrote: | Using this and a 2GB ccache my compiles are considerably faster (KDE version and release bumps in around 2 hours on a 1GHz Athlon) |
What's this ccache you speak of? |
|
Back to top |
|
|
TheEternalVortex Apprentice
Joined: 15 Oct 2002 Posts: 207 Location: San Jose, CA
|
Posted: Sat Nov 16, 2002 6:12 am Post subject: |
|
|
emerge ccache
It works quite well, it caches compiles making building the same app more than once a LOT faster. _________________ -- Andy |
|
Back to top |
|
|
dreamer3 Guru
Joined: 24 Sep 2002 Posts: 553
|
Posted: Sat Nov 16, 2002 6:32 am Post subject: |
|
|
TheEternalVortex wrote: | emerge ccache
It works quite well, it caches compiles making building the same app more than once a LOT faster. |
Does it work automatically after I emerge it? |
|
Back to top |
|
|
puddpunk l33t
Joined: 20 Jul 2002 Posts: 681 Location: New Zealand
|
Posted: Sat Nov 16, 2002 7:47 am Post subject: |
|
|
IIRC you just have to add a small line of text at the bottom of /etc/make.conf.
It tells you how after you finished building it
EDIT: Scratch that:
Code: | * To use ccache, add /usr/bin/ccache to your path before /usr/bin.
* Portage 2.0.6+ will automatically take advantage of ccache with no additional steps.
* If this is your first install of ccache, type something like this to set a maximum
* cache size of 2GB (or whatever you desire):
* # /usr/bin/ccache/ccache -M 2G |
|
|
Back to top |
|
|
dreamer3 Guru
Joined: 24 Sep 2002 Posts: 553
|
Posted: Sat Nov 16, 2002 7:52 am Post subject: |
|
|
puddpunk wrote: | IIRC you just have to add a small line of text at the bottom of /etc/make.conf.
It tells you how after you finished building it |
This ONLY helps if you're compiling something for the SECOND time, right? |
|
Back to top |
|
|
puddpunk l33t
Joined: 20 Jul 2002 Posts: 681 Location: New Zealand
|
Posted: Sat Nov 16, 2002 7:57 am Post subject: |
|
|
yes, unless someone wants to lend you their ~/.ccache dir (not recommended!!!) |
|
Back to top |
|
|
|