| View previous topic :: View next topic |
| Author |
Message |
Cazzantonio
Bodhisattva
Joined: 20 Mar 2004
Posts: 4514
Location: Somewere around the world
|
 Posted: Fri Dec 24, 2004 1:34 pm Post subject: |
 |
|
with 1gb ram and several gigabytes of hd space to use as swap memory do you think it' safe to use | Code: | | PORTAGE_TMPDIR=/tmp |
?
Did someone have some problems using tmpfs to compile programs? Consider that a some of ram is used in compiling, if ram gets filled by tmpfs the system could get much slower...
I wonder if there is an option lo limit the amount of phisical ram used without limiting the usage of swap space
_________________
Any mans death diminishes me, because I am involved in Mankinde; and therefore never send to know for whom the bell tolls; It tolls for thee.
-John Donne |
|
| Back to top |
|
 |
Gherald
Veteran
Joined: 23 Aug 2004
Posts: 1399
Location: CLUAConsole
|
| Posted: Fri Dec 24, 2004 2:45 pm Post subject: |
|
|
| but tmpfs on swap is slower than reiserfs on disk |
|
| Back to top |
|
|
Jinidog
Guru
Joined: 26 Nov 2003
Posts: 593
Location: Berlin
|
| Posted: Fri Dec 24, 2004 10:31 pm Post subject: |
|
|
Okay, I made an emerge -e world using tmpfs for portage.
Nearly every package compiled around 30% faster than they compiled in earlier runs. (confirmed by genlop)
It's not 100% sure, because normaly compiling wasn't the only task of my PC (I surfed besides that) but this time it was.
I have 512 MB RAM and I closed everything during compiling and just had KDE up running.
_________________
Just unused Microsoft-Software is good Microsoft-Software |
|
| Back to top |
|
|
drphibes
Guru
Joined: 30 Nov 2004
Posts: 432
|
| Posted: Sat Dec 25, 2004 4:26 am Post subject: |
|
|
how much space do you need to emerge a big package like xorg or kde? i assume one would need to limit tmpfs sizes at mount time in order to guarantee you don't blow out your virtual memory....e.g. 512M max for a 1G physical memory box with, say a 1G swap. Is 512M enough space to emerge the big packages? autoclean must be "yes" right or your hosed?
doc |
|
| Back to top |
|
|
hds
Advocate
Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]
|
| Posted: Sat Dec 25, 2004 4:34 am Post subject: |
|
|
| Cazzantonio wrote: | with 1gb ram and several gigabytes of hd space to use as swap memory do you think it' safe to use | Code: | | PORTAGE_TMPDIR=/tmp |
?
|
emerge UT2004 
btw.. if you are using VMWARE, you cant do that either. |
|
| Back to top |
|
|
ahorn
Guru
Joined: 01 Nov 2004
Posts: 366
Location: Ol' europe
|
| Posted: Fri Jan 21, 2005 10:53 pm Post subject: |
|
|
if /tmp and /var/tmp is at tmpfs, it is cleaned out at boot. but gentoo cleans it at the booting process anyway.
how can i disable this to get some more speed at boot?
modify /etc/init.d/bootmisc, yes?
look here in bootmisc:
| Code: | #
# Clean up /tmp directory
#
if [ -z "${CDBOOT}" ]
then
ebegin "Cleaning /tmp directory"
#rm -f /tmp/.X*-lock /tmp/esrv* /tmp/kio* /tmp/jpsock.* /tmp/.fam* &>/dev/null
#rm -rf /tmp/.esd* /tmp/orbit-* /tmp/ssh-* /tmp/ksocket-* /tmp/.*-unix &>/dev/null
# Make sure our X11 stuff have the correct permissions
mkdir -p /tmp/.{ICE,X11}-unix &>/dev/null
chown root.root /tmp/.{ICE,X11}-unix &>/dev/null
chmod 1777 /tmp/.{ICE,X11}-unix &>/dev/null
eend 0
fi
|
as you see
| Code: | | chmod 1777 /tmp/.{ICE,X11}-unix &>/dev/null |
it is not necessary to mount tmpfs /tmp with mode=1777
_________________
Too many connections. Please try again later. |
|
| Back to top |
|
|
ba747heavy
Apprentice
Joined: 11 Aug 2003
Posts: 216
Location: New Mexico, USA
|
| Posted: Fri Jan 21, 2005 11:23 pm Post subject: |
|
|
Hmm, my computer likes to lock up, and it happens with any combination of programs running and not running. Now I wonder if the fact that my /tmp is a tmpfs has anything to do with it. Any way to know? I guess I can mount my /tmp as a normal partition and see what happens.
_________________
Fred Clausen
"leet [speak] is a cry for help from a shallow mind" - Doomwookie Jan 05 |
|
| Back to top |
|
|
beandog
Bodhisattva
Joined: 04 May 2003
Posts: 2072
Location: /usa/utah
|
| Posted: Fri Jan 21, 2005 11:32 pm Post subject: |
|
|
| Jinidog wrote: | strange.
On a system with 512 MB of RAM I tested the performance of being /var/tmp/portage a tmpfs.
The results:
without tmpfs:
real 2m41.081s
user 1m48.505s
sys 0m20.697s
with tmpfs:
real 2m58.270s
user 1m50.086s
sys 0m21.272s
Concluding the filesystem in the memory gave worse performance than having a normal Reiser3.6 Filesystem on the harddisk.
CCache was not activated.
I said it, strange. |
I got the same result. Oh well.
Still, its cool to use as a replacement for /tmp at least.
Steve
_________________
If it ain't broke, tweak it. dvds | blurays | blog | wiki |
|
| Back to top |
|
|
MighMoS
Guru
Joined: 24 Apr 2003
Posts: 416
Location: @ ~
|
| Posted: Sat Jan 22, 2005 3:20 am Post subject: |
|
|
I put /var/run in a tmpfs file system, so that should I lose power (we had power work being done for a while) or the machine freeze b/c of buggy NVidia drivers, programs wont get confused when they see their old PID files.
_________________
jabber: MighMoS@jabber.org
localhost # export HOME=`which heart` |
|
| Back to top |
|
|
Master One
l33t
Joined: 25 Aug 2003
Posts: 754
Location: Austria
|
| Posted: Mon Feb 14, 2005 7:05 pm Post subject: |
|
|
After reading this very interesting thread, I just wondered, how you guys would use tmpfs if you had a new IBM ThinkPad T42p with 2 GB RAM and a 4 GB swap partition. I'd like to setup my upcoming machine in the most efficient way. BTW Any info, if tmpfs operates well with hibernation of swsuspend2?
_________________
Las torturas mentales de la CIA |
|
| Back to top |
|
|
Jinidog
Guru
Joined: 26 Nov 2003
Posts: 593
Location: Berlin
|
| Posted: Mon Feb 14, 2005 8:28 pm Post subject: |
|
|
The only way is to benchmark, what is faster for you.
Concluding, I see no sence in mounting a temporary filesystem for the portage-working directory.
If there is enough free RAM, everything what compiling needs is cached in the RAM. It's the same like a filesystem in the RAM.
So, if there is not enough RAM, the cached files are removed from memory.
If it is a temporary filesystem and the memory is full, it has to be swapped, what needs time.
_________________
Just unused Microsoft-Software is good Microsoft-Software |
|
| Back to top |
|
|
hds
Advocate
Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]
|
| Posted: Mon Feb 14, 2005 11:02 pm Post subject: |
|
|
| Master One wrote: | | 4 GB swap partition |
whatever came to your mind to setup a 4GB swap? false dope? |
|
| Back to top |
|
|
Master One
l33t
Joined: 25 Aug 2003
Posts: 754
Location: Austria
|
| Posted: Tue Feb 15, 2005 9:00 am Post subject: |
|
|
| hds wrote: | | whatever came to your mind to setup a 4GB swap? false dope? |
You are right, that would indeed be a little overkill, was sticking with the general rule "twice the RAM"...
As I want to use hibernation on that machine, swap should be at least 2048 MB (or larger? didn't find the time for much reading on this subject with swsusp2 & hibernation).
_________________
Las torturas mentales de la CIA |
|
| Back to top |
|
|
Erlend
Guru
Joined: 26 Dec 2004
Posts: 493
|
| Posted: Thu Feb 17, 2005 2:54 pm Post subject: |
|
|
I've been curious about this for a while.
I notice that I have a gig or so in /tmp - it never seems to get automatically deleted. I'm not sure if I can delete all of it though.
I would guess that if people are having slower compile in tmpfs it's because they don't have enough available ram. Listen to the drive and see if it is using the swap partition?
I was wondering about putting and ext2 filesystem on /tmp /var/tmp and /boot, since this has very little overhead, nice and fast and it doesn't matter if any of this is corrupted.
Any ideas?
Thanks,
Erlend |
|
| Back to top |
|
|
hds
Advocate
Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]
|
| Posted: Thu Feb 17, 2005 5:43 pm Post subject: |
|
|
| Erlend wrote: |
notice that I have a gig or so in /tmp - it never seems to get automatically deleted. I'm not sure if I can delete all of it though.
|
init 1
delete contents of /tmp
init 3 (or whatever your usual runlevel is) - voila
btw.. using ext2 for /boot and / over here, and ext3 for /data (just because i need largefile support [>4GB], otherwise i would use ext2 there as well).
ext2 is still the fastest, while takeing least CPU, IMHO.
and, if you ever f*cked up a reiserfs - sheesh, you will never use reiser again.
if you get hardware errors on your hdd (and this will hapen sooner or later to anyone of us), bet you run ext2 or ext3, because you will no be able to restore anything from your reiser.
beware of reiser, if your S.M.A.R.T will get full!! and until you realize hat, it is too late aleady ;( |
|
| Back to top |
|
|
Erlend
Guru
Joined: 26 Dec 2004
Posts: 493
|
| Posted: Thu Feb 17, 2005 6:23 pm Post subject: |
|
|
| Quote: | init 1
delete contents of /tmp
init 3 (or whatever your usual runlevel is) - voila |
Thanks for that.
| Quote: | | you will no be able to restore anything from your reiser. |
Is that just because it is too complicated, and no recovery tools exist?
| Quote: | | beware of reiser, if your S.M.A.R.T will get full!! and until you realize hat, it is too late aleady ;( |
Not sure what you mean I'm afraid?
Erlend |
|
| Back to top |
|
|
hds
Advocate
Joined: 21 Aug 2004
Posts: 2629
Location: Sprockhoevel [GER]
|
| Posted: Thu Feb 17, 2005 6:53 pm Post subject: |
|
|
| Erlend wrote: |
| hds wrote: | | you will no be able to restore anything from your reiser. |
Is that just because it is too complicated, and no recovery tools exist? |
no, recovery tools do exist, but infact of a hardware defect of the HDD, you are simply lost. the recovery tools will fail, at least they did here (reiser3, that was).
they will try to move data, or write a new superblock (depends on the options you give), and fail, because the next block they try will get unwritable as well. this gets into a ratrace, marking blocks as bad all and all over ;(
| hds wrote: | | beware of reiser, if your S.M.A.R.T will get full!! and until you realize hat, it is too late aleady ;( |
| Erlend wrote: |
Not sure what you mean I'm afraid?
|
i am talking about the harddrives internal "bad block table". if that gets full, an IDE drive will get more and more corrupted with bad blocks at any write acess you do.
in this case, one prefers to save as much data as possible, but reiser will also deny readaccess in this case, because its journal is corrupted!
if you come from DOS, you might be familiar with "lost+found" directory, thats what "checkdisk" did in the very old days.
btw: this is not against reiser in general, i guess this will happen with any journaling filesys infact of hardware defects.
and btw: yes, i do backups, but you are always caught if you did something you wouldnt like to loose. shit happens  |
|
| Back to top |
|
|
Erlend
Guru
Joined: 26 Dec 2004
Posts: 493
|
| Posted: Thu Feb 17, 2005 10:22 pm Post subject: |
|
|
Thanks. I didn't know any of that.
I guess one advantage of having /tmp /var/tmp in RAM is that it will increase the lifetime of your hard disk - writing/reading small files like crazy can't be particularly good for it. On that note, is "emerge -e world" "bad" for your CPU? I'm just wondering if making your computer compile for 30 odd hours is going to affect it's performance/life?
Also, could one not use device-mapper:
echo "0 X linear /dev/ram0 0 /dev/hda4 0" | dmsetup create tmpHybrid
...
mount /dev/mapper/tmpHybrid
So that the first 50MB of /tmp is RAM, and the rest ext2 space?
Erlend |
|
| Back to top |
|
|
Master One
l33t
Joined: 25 Aug 2003
Posts: 754
Location: Austria
|
| Posted: Fri Feb 18, 2005 12:17 pm Post subject: |
|
|
| MighMoS wrote: | | I put /var/run in a tmpfs file system, so that should I lose power (we had power work being done for a while) or the machine freeze b/c of buggy NVidia drivers, programs wont get confused when they see their old PID files. |
This makes no sense at all, because /var/run get's cleaned on every boot anyway, see /etc/init.d/bootmisc:
| Code: | ebegin "Cleaning /var/lock, /var/run"
rm -rf /var/run/console.lock /var/run/console/*
if [[ -z ${CDBOOT} ]] ; then
#
# Clean up any stale locks.
#
find /var/lock -type f -print0 | xargs -0 rm -f --
#
# Clean up /var/run and create /var/run/utmp so that we can login.
#
for x in $(find /var/run/ ! -type d ! -name utmp ! -name innd.pid ! -n$
local daemon=${x##*/}
daemon=${daemon%*.pid}
# Do not remove pidfiles of already running daemons
if [[ -z $(ps --no-heading -C "${daemon}") ]] ; then
if [[ -f ${x} || -L ${x} ]] ; then
rm -f "${x}"
fi
fi
done
fi |
_________________
Las torturas mentales de la CIA |
|
| Back to top |
|
|
markandrew
Apprentice
Joined: 04 Feb 2004
Posts: 176
Location: Manchester, UK
|
| Posted: Fri Feb 18, 2005 3:00 pm Post subject: |
|
|
I had /tmp and /var/tmp as tmpfs for a while but went back to normal fs (reiser) after a while as i saw no noticeable speed increase in general usage or during compiling, and emerging large packages, like xorg or openoffice, would fill out the /var/tmp dir and i'd have to start over. dedicating more memory, i decided, wasn't worth it (got 1G ram).
I also had, for a while, my /var/logs dir as tmpfs, with it being read from disk on boot and written back on shutdown. however after a couple of crashes (non related) i realized that if the machine didn't shutdown properly, i lost all my logs for that session... and thus couldn't figure out why the crashes were happening.
in the end i decided it was a nice idea, but didn't give any real benefits. |
|
| Back to top |
|
|
Erlend
Guru
Joined: 26 Dec 2004
Posts: 493
|
| Posted: Fri Feb 18, 2005 8:11 pm Post subject: |
|
|
I've always wondered it having /etc/init.d in ram at boot would speed things up. I imagine it wouldn't make much difference though?
Just a though, /var is temporary in that it is meant for files which are non-essential, and which don't change very often: surely then that could be tmpfs or some low-overhead filesystem (ext2).
Erlend |
|
| Back to top |
|
|
kallamej
Administrator
Joined: 27 Jun 2003
Posts: 4975
Location: Gothenburg, Sweden
|
| Posted: Fri Mar 25, 2005 8:07 am Post subject: |
|
|
| Erlend wrote: | | Just a though, /var is temporary in that it is meant for files which are non-essential, and which don't change very often: surely then that could be tmpfs or some low-overhead filesystem (ext2). |
Do not use tmpfs for /var, portage and other programs depend on it surviving a reboot.
_________________
Please read our FAQ Forum, it answers many of your questions.
irc: #gentoo-forums on irc.libera.chat |
|
| Back to top |
|
|
lost+found
Guru
Joined: 15 Nov 2004
Posts: 509
Location: North~Sea~Coa~s~~t~~~
|
| Posted: Tue Apr 19, 2005 1:10 pm Post subject: |
|
|
This is how I made a virtual Test/Guest Account in tmpfs /tmp.
| Code: | # useradd test -m -G users,audio,tty -s /bin/bash -d /tmp/test
# passwd test |
/etc/conf.d/local.start:
| Code: | cp -R /etc/skel /tmp
rename skel test /tmp/skel
chmod 0700 /tmp/test
chown -R test:users /tmp/test |
Restart... |
|
| Back to top |
|
|
mr.ed
Tux's lil' helper
Joined: 30 Nov 2004
Posts: 75
|
| Posted: Sun Apr 24, 2005 1:56 pm Post subject: |
|
|
K, here r my two cents...
| Quote: | [15:29][root:-> /]@: df
/dev/sda3 45G 5.8G 39G 14% /
udev 1005M 600K 1004M 1% /dev
tmpfs 1.0G 1.4M 1023M 1% /dev/shm
/dev/sdb1 66G 24G 43G 36% /home
|
| Quote: | 0 lrwxrwxrwx 1 root root 12 Apr 12 15:25 /tmp -> /dev/shm/tmp
0 lrwxrwxrwx 1 root root 12 Apr 12 15:25 /var/tmp -> /dev/shm/tmp |
This is my fstab-line for /dev/shm (nothing else!)
| Quote: | | tmpfs /dev/shm tmpfs defaults,size=1G 0 0 |
As u can see my tmp-dir is mounted at /dev/shm (which is the standard tmpfs). I noticed this thread before and thought if it would be possible to redirect all of your tmp-dirs to /dev/shm and thus move all of your workfiles to tmpfs... After a bit of searching round i came to the conclusion it can be done and it's a real speedgain to. At first i tried the fstab solution, but it gave 2 much trouble (mount bind and stuff). But then i realised a symlink would do exact the same thing!!!
Basically i just deleted all of the tmp-dirs first and then created the dir /dev/shm/tmp. Then I chmod 1777 /dev/shm/tmp (NOT /dev/shm) and symlinked the tmp-dirs to this folder.
The only downside is off course u gotta have enough ram to compile larger progs like xorg, kde (for OOo or anything else larger then my tmpfs i remount /var/tmp/portage to a partition). But then u would still have the problem of your ram filling up rather quickly and not getting cleaned... My solution was emerging tmpwatch and modifying the cronscript in such a way that it cleans out my /var/tmp/portage every hour of files that haven't been accessed for one hour and cleans out /tmp every 8 hours. I added the script to cron to be run every half hour, so the cache never outgrows my tmp!!!
The second problem was that the /dev/shm/tmp get's deleted off course at every reboot, so i added this to /etc/init.d/localmount:
| Quote: | start() {
# Make tmp directories for mounting /etc/fstab.
mount /dev/shm
mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
# Mount local filesystems in /etc/fstab.
ebegin "Mounting local filesystems" |
Any1 paying attention will see that this combination creates a tmp-dir IN tmpfs, not a dir mounted on tmpfs (cause it allready exists with /dev/shm, see your own df -h)
If some1 has questions bout it, i'ld be willing to set up some sort of guide to how to do it...
P.s.: I got 2 gigs of ddr2 ram and i can leave my comp on for days in a row if i'ld like, it runs the same as if temp was normally mounted at a filesystem, but faster  ... This setup i've been using for 3 weeks now and never got into trouble with tmp filling up!! (Never grew to more then 75%)
_________________
(Ex-Pornstar)
Last edited by mr.ed on Sun Apr 24, 2005 9:24 pm; edited 1 time in total |
|
| Back to top |
|
|
lost+found
Guru
Joined: 15 Nov 2004
Posts: 509
Location: North~Sea~Coa~s~~t~~~
|
| Posted: Sun Apr 24, 2005 6:13 pm Post subject: |
|
|
| mr.ed wrote: | | ...but it gave 2 much trouble (mount bind and stuff)... |
The mount --bind thing is easy... 
This makes /dev/shm and /tmp share one tmpfs.
/etc/conf.d/local.start:
| Code: | cp -Rp /tmp /dev/shm
mount --bind /dev/shm/tmp /tmp |
 cp instead of mkdir, this avoids X11 crash.
/etc/fstab:
| Code: | | none /dev/shm tmpfs nodev,nosuid,noexec,noatime,size=256M 0 0 |
Restart... |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
