Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Mount / in RAM and load apps instantly
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Wed Feb 15, 2006 1:43 pm    Post subject: Reply with quote

no, you don't have to edit any such list. It's automatic and statistical. All you have to do is turn it on.
Back to top
View user's profile Send private message
brazzmonkey
Guru
Guru


Joined: 16 Jan 2005
Posts: 372
Location: between keyboard and chair

PostPosted: Wed Feb 15, 2006 2:02 pm    Post subject: Reply with quote

allright, this should be ok then. i also added it to default runlevel, because i saw an entry in rc-update show.
if it uses statistics, i suppose this should take some time to get some measurable effect...

thanks a lot for your lightning-fast replies !!
Back to top
View user's profile Send private message
Centinul
Apprentice
Apprentice


Joined: 28 Jul 2005
Posts: 232

PostPosted: Sat Mar 25, 2006 3:05 am    Post subject: Reply with quote

I run a linux based firewall. It is a celeron (mendocino) 550mHz with 256MB of RAM and a 4Gb HD. I was thinking about using this as a security measure on the firewall that way if someone got in and modified files they would be reset on reboot. Is this a viable method? Advice needed. I really don't understand how this protects your system. I would also like someone to explain that to me please.

Here is some output for size of certain folders.

Code:

139M /usr/lib
5.9M /lib
19M /usr/bin
3.6M /usr/sbin
5.2M /sbin


When the system is at idle. I have the following output from "free -m"
Code:

               total     used free shared buffers cached
Mem:           248       244    3       0        132       5
-/+ buffers/cache:       106    142
Swap:          494       0    494


Any thoughts would be appreciated. Thanks.
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Sat Mar 25, 2006 3:22 am    Post subject: Reply with quote

I think you are confusing two very distinct ideas:

1) pre-cache files which are accessed frequently. This will increase system responsiveness. This is mostly what is discussed in tis thread.
2) having a read-only root with RAM-mounted (tmpfs) access for read/write access which is lost on reboot.

The second can be found elsewhere. I bet the Gentoo securiy howto or the gentoo-wiki has info on this.
Back to top
View user's profile Send private message
Centinul
Apprentice
Apprentice


Joined: 28 Jul 2005
Posts: 232

PostPosted: Sat Mar 25, 2006 6:10 am    Post subject: Reply with quote

adsmith ---
Thanks for the info. I was wondering if you could direct me to a location of a howto for a read only /? I can't find anything in the Security Handbook, Gentoo Wiki or the forums. Thanks!
Back to top
View user's profile Send private message
adsmith
Veteran
Veteran


Joined: 26 Sep 2004
Posts: 1386
Location: NC, USA

PostPosted: Sat Mar 25, 2006 2:27 pm    Post subject: Reply with quote

Here, at gentoo-wiki:
http://gentoo-wiki.com/HOWTO_Read-only_root_filesystem
This was the first hit on google for "read only root flesystem linux"
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Sun Mar 26, 2006 12:44 am    Post subject: Reply with quote

that got me thinking... you could also set up a script that remounts / after boot time (say, trigger it from local.start). you'd still want some files in /etc to be writable, so the script would have to take that into account. could be simplier though
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Wed Apr 05, 2006 3:26 pm    Post subject: Reply with quote

well, i found a sort of cheap workaround so you can "preload" apps into RAM. really, i'd only need it for firefox and thunderbird, as everything else i use loads fast enough for my tastes.

behold: Kdocker

http://kdocker.sourceforge.net/

"KDocker will help you dock any application into the system tray. This means you can dock openoffice, xmms, firefox, thunderbird, anything!"

the idea is to have your WM load the app as a system tray process. (it's not just for KDE, though it requires QT.) really the app is actually running, the idea is you click on the icon and it comes up immediately. if you have enough RAM for the app, should work fine without chunking up your system. best for a few specific apps (not everything in /usr/bin 8O )
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
Kragen
Tux's lil' helper
Tux's lil' helper


Joined: 28 Mar 2006
Posts: 77

PostPosted: Fri Apr 07, 2006 11:40 pm    Post subject: Reply with quote

I've only read the first page, so I'm sorry if this has been mentioned before but...

Shouldnt there be a way of loading commonly used libraries into ram anyway? I mean, doesnt windows do this? Surely what's really needed is a program / system that monitors the useage of different libraries, decides how much and what should be loaded into ram based on the usage of different libraries and available ram, and loads it all for you transparantly.
Back to top
View user's profile Send private message
Kragen
Tux's lil' helper
Tux's lil' helper


Joined: 28 Mar 2006
Posts: 77

PostPosted: Sat Apr 08, 2006 2:03 pm    Post subject: Reply with quote

ok - this readahead is pretty much exacltay what I was thinking of. Does it make any difference?
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2160
Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Thu Jun 29, 2006 3:31 am    Post subject: Reply with quote

OK, I haven't read this entire thread so this may have already been brought up. Won't a simple script in local.start that does "cat $NAME_OF_FILE_TO_PRELOAD >> /dev/null" have the exact same effect of preloading the file into RAM without having to do all that magic with a ramdisk?

I use this on some of my machines for mozilla to load faster and it works like a charm. PLUS, you don;t lose any of your RAM space to the ramdisk (which has a set amount of RAM it uses) because the least used apps just get swapped out freeing your RAM for the stuff in memory that you are actually using.
_________________
Gentoo: it's like wiping your ass with silk.
Back to top
View user's profile Send private message
killercow
Tux's lil' helper
Tux's lil' helper


Joined: 29 Jan 2004
Posts: 86
Location: Netherlands

PostPosted: Fri Jun 30, 2006 8:59 am    Post subject: Reply with quote

agrees,

Any file loaded of disk will be stored in ram for as long a possbile.

On my 4gb box, i can actually see whole movies trickle into ram, while i download/decompress them.

After something like 6 movies, ram is full, and the first one is purged again. (but only partially), Thus it automatically keeps everything as speedy as possible.

This is also the elusive memmory usage, linux newbee's complain about.
Back to top
View user's profile Send private message
Zentoo
Apprentice
Apprentice


Joined: 18 Nov 2002
Posts: 195
Location: /dev/console

PostPosted: Wed Aug 02, 2006 12:15 pm    Post subject: Found a way to list needed files to cache ! Reply with quote

Hey, i'm pretty interesting to cache my applications in ram at boot time so i 've read the thread.
But as a lot of us, my /usr/lib* is really huge for my RAM so the question is:

How choose files that needs to be cached ?

i think i've found an easyway to do it:

you need to:

Code:
emerge sys-process/lsof 
(Lists open files for running Unix processes)

then launch manually all the applications that you want speed up and just type the following command in a shell:

Code:
  lsof -F | sed "s/^n//g" | grep -v "^c" | egrep "^/bin|^/lib|^/sbin|^/usr" | sort | uniq


and there you have the list of files open for your process, you need just to redirect it in a file and it's done ! ;)



NOTE: i think we don't catch all files involved at starting time so i'm looking for a way to list all the files used for a time lap. If anyone have an idea ?
_________________
Kernel 5.14.15-zen | Gcc 11.2 | Glibc 2.34
Core i7 6700K @ 4.6GHz | 32Gb
ACCEPT_KEYWORDS="~amd64"
CFLAGS="-march=native -O2 -pipe"
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 3505

PostPosted: Tue Jan 30, 2007 1:33 pm    Post subject: Reply with quote

I just came across this thread, while searching for something else. As I was skimming it, another "opportunity" occurred to me. For the ext2 filesystem, there is a relatively new option, "execute in place". (CONFIG_EXT2_FS_XIP) Think about it for a minute... You've just loaded your executable files into RAM, so you can get at them quickly. When it's time to run something like firefox, you grab a copy from filesystem in RAM, and load it into... RAM. You've now got 2 copies of firefox in RAM, one as a file and one as executing code. Seems to me that this is the situation XIP was made for.

Beyond that, take a look at what is really happening here. As someone has mentioned, by default Linux caches files. The problem is that it has no idea what file you're about to ask for, but there is the underlying assumption that if you've asked for it once, you will likely ask for it, again. That's what a cache does. What you're really doing is saying, "I'm smarter than a cache, because I *know* what I'm going to need, and will preload it into RAM." But you've just rather indiscriminately put practically *everything* into RAM.

You can be smarter. How about a directory structure with /ram, containing /ram/lib and /ram/bin, maybe /ram/sbin? Take the stuff you *really* want to be fast, and copy it there at initialization. Change your path to put /ram/bin ahead of /usr/bin, and change /etc/ld.so.conf to upt /ram/lib ahead of /use/lib. With a little thought, this can be combined with XIP mentioned above.

By the way, it's probably counter productive to put something *most* used like bash into /ram/bin. Most likely it will remain RAM resident for the duration of most boots, and copying the file into RAM is a waste of space.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
daveisgreat
n00b
n00b


Joined: 13 Nov 2004
Posts: 19
Location: Kent

PostPosted: Wed Jan 31, 2007 8:38 pm    Post subject: Reply with quote

Nice HOWTO! Just what I came looking for! Cheers!
_________________
Hello again!
Back to top
View user's profile Send private message
Bono
n00b
n00b


Joined: 18 Feb 2005
Posts: 24

PostPosted: Mon Mar 12, 2007 9:11 am    Post subject: Reply with quote

Hello,

Concerning this How to, how do you manage emerge working well after that ? I mean, it will install libraries in the ramdisk for instance and they won't be anymore on the disk ?

Thanks in advance,
Marc
Back to top
View user's profile Send private message
abfluss_bombe
n00b
n00b


Joined: 09 Feb 2007
Posts: 10

PostPosted: Wed Apr 11, 2007 9:49 pm    Post subject: Reply with quote

perhaps you could use unionfs for that? i havent looked deeply in that but it has somehow to be possible to read from ramdisk but write on harddisk. ok the update if files are changed could be a problem.
Back to top
View user's profile Send private message
thebigslide
l33t
l33t


Joined: 23 Dec 2004
Posts: 792
Location: under a car or on top of a keyboard

PostPosted: Fri Apr 20, 2007 11:26 pm    Post subject: Reply with quote

Bono wrote:
Hello,

Concerning this How to, how do you manage emerge working well after that ? I mean, it will install libraries in the ramdisk for instance and they won't be anymore on the disk ?

Thanks in advance,
Marc


Hi. The easiest way is to use a script to update the tarball that is unpacked to populate the ramdrive bootup. With a little work you can easily make this an init script.

Perhaps this howto could use some revision.
Back to top
View user's profile Send private message
ssam
n00b
n00b


Joined: 24 Nov 2005
Posts: 13

PostPosted: Sat Jun 30, 2007 2:00 pm    Post subject: Reply with quote

have you looked at preload http://sourceforge.net/projects/preload

Quote:
preload is an adaptive readahead daemon. It monitors applications that users run, and by analyzing this data, predicts what applications users might run, and fetches those binaries and their dependencies into memory for faster startup times.


there is also a releated summer of code project http://code.google.com/soc/2007/ubuntu/appinfo.html?csaid=8EDA2B217C83972
http://code.google.com/p/prefetch/
Back to top
View user's profile Send private message
NaiL
Apprentice
Apprentice


Joined: 13 Feb 2003
Posts: 228
Location: Spain/BCN

PostPosted: Sat Jun 30, 2007 2:32 pm    Post subject: Reply with quote

May be all this stuff can be combined with this tip:

https://forums.gentoo.org/viewtopic-t-465367.html
Back to top
View user's profile Send private message
kusi
n00b
n00b


Joined: 11 Jun 2005
Posts: 21

PostPosted: Sun Mar 02, 2008 2:16 pm    Post subject: Reply with quote

I mounted my /var/tmp/paludis as tmpfs and did a performance analysis: I emerged digikam

my ram drive:
Code:
mount -t tmpfs tmpfs -o size=2000M /var/tmp/paludis


I emerged digikam twice
Code:
time paludis -i digikam


w/o ramdrive
real 15m12.167s
user 10m27.595s
sys 8m59.962s

with ramdrive
real 15m9.048s
user 10m30.951s
sys 8m53.873s

As you can see, the speed benefit of using a ramdrive is marginal. Did somebody experience the same? Is this what you have to expect with modern hardware? I use a 2.2ghz core 2 duo, 4gb ram

Kusi
Back to top
View user's profile Send private message
micr0c0sm
Tux's lil' helper
Tux's lil' helper


Joined: 29 Oct 2005
Posts: 148
Location: New York

PostPosted: Tue Mar 04, 2008 1:04 am    Post subject: Reply with quote

using -pipe pretty much makes sure everything is in ram anyway, so there should be no compilation speedup. Throughput shouldn't increase (unless you have a truly fast usb drive and really slow hard drive), but responsiveness is much, much better since there is no spinup or seeking.
Back to top
View user's profile Send private message
spupy
Tux's lil' helper
Tux's lil' helper


Joined: 08 May 2007
Posts: 102
Location: Germany

PostPosted: Fri Nov 14, 2008 11:52 pm    Post subject: Reply with quote

This is what I did:
I renamed /usr/bin to abin. Created a dir named bin and mounted it into ram with:
Code:
mount -t tmpfs -o size=300m none /usr/bin > /dev/null 2>&1

Moved the stuff from abin to bin.
Did the same for /usr/lib.

Unfortunately, this did not decrease the cold-boot times of programs. Does this command really mount the folder in ram, because it looked like it didn't...
_________________
Make install - not war!
Back to top
View user's profile Send private message
robak
Apprentice
Apprentice


Joined: 14 Jan 2004
Posts: 209
Location: Germany

PostPosted: Tue Dec 30, 2008 2:13 pm    Post subject: Reply with quote

Hi guys!

i know that this thread is old but i have a little problem.
i put the whole filesystem into one file excluding etc and var dirs. the problem is that i get tons of "No such file or directory" errors while unpacking the fs.tar
and no errors while creating it.

Code:

.....
tar: usr/sbin/paperconfig: Cannot open: No such file or directory
tar: usr/sbin/ck-log-system-start: Cannot open: No such file or directory
tar: usr/sbin/rpcinfo: Cannot open: No such file or directory
tar: usr/sbin/accept: Cannot open: No such file or directory
tar: usr/sbin/in.rshd: Cannot create symlink to `rshd': No such file or directory
tar: usr/sbin/useradd: Cannot open: No such file or directory
tar: usr/sbin/partx: Cannot open: No such file or directory
.....



can someone help me?


greetings robak
Back to top
View user's profile Send private message
robak
Apprentice
Apprentice


Joined: 14 Jan 2004
Posts: 209
Location: Germany

PostPosted: Tue Dec 30, 2008 2:58 pm    Post subject: Reply with quote

i figured out, what the problem is:

my ramdisk has 4G space and should be big enough to store the whole fs (which has 2,9G) but somehow i run out of free-space. does anybody know how to fix that?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 5 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum