Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Mount / in RAM and load apps instantly
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
rutski89
Guru
Guru


Joined: 14 Mar 2005
Posts: 468
Location: United States N.Y.

PostPosted: Tue Mar 22, 2005 1:26 am    Post subject: Reply with quote

Would mounting /usr/lib, /lib etc... etc... make a screen video capture program such as xvidcap fly as well? Or is it really ONLY the opening of the program that speeds up.
Back to top
View user's profile Send private message
unseen-enigma
n00b
n00b


Joined: 14 Feb 2004
Posts: 31

PostPosted: Fri Mar 25, 2005 1:54 am    Post subject: Alternative Method - Select Apps Reply with quote

I did a similar thing using a slightly different method (sorry I dont have step by step but its not a big procedure). The problem with this procedure for many systems is simple: Too many applications / libraries. So I ported over a basic chroot build script (takes core libs and ldd of a given app stores in a directory). I loaded openoffice(word-like part only), firefox, xorg, xine, and several (about 20) core kde applications into the /ramdisk I made under /ramdisk/bin and /ramdisk/lib (added to path). A simple init script loads this into ram on boot. If i need to add a application I can simply add it to my (new) /etc/ramapps file. My system is speedy and it only cost about 250mB of ram instead of the 1-2gb of many others on this forum. This approach(and the parent) is great for a terminal system far more than a single user system as you can reap far greater benifit from precaching common application in ram when you can be reasonably sure 90% of them will actually benifit performance rather then clog ram. Anyone implementing this method please remember to put the /ramdisk/bin first in the PATH and similarly for lib. If you dont want to or cant rely on the path variable in the way your applications launch you can try bind mounting each applicatiion and library over its / based counterpart but I have no idea what that many bind mounts might do to performance. You could copy and bindmount /etc but I didnt see the need and I think it would be a pain to keep umounting it whenever I need to install or update my system
Back to top
View user's profile Send private message
rutski89
Guru
Guru


Joined: 14 Mar 2005
Posts: 468
Location: United States N.Y.

PostPosted: Fri Mar 25, 2005 3:29 pm    Post subject: Reply with quote

Awasom, I'm defeintly goning to get on this soon and set things up. I've got 512 ram, I've got to do some "watch -n .1 freem -m" monitoring on the buffers first to see how much mounting my ram can take. What is the "correct" way to perminently change your $PATH?
Back to top
View user's profile Send private message
unseen-enigma
n00b
n00b


Joined: 14 Feb 2004
Posts: 31

PostPosted: Fri Mar 25, 2005 11:15 pm    Post subject: PATH Changes Reply with quote

/etc/profile is what i normally use
Back to top
View user's profile Send private message
ruben
Guru
Guru


Joined: 04 Jul 2003
Posts: 462

PostPosted: Tue Mar 29, 2005 10:51 pm    Post subject: unionfs Reply with quote

thebigslide,

Actually, i've been thinking before about doing something similar to what you present in this howto. I use a laptop with a 4200rpm harddisk, and starting applications can take a long time. Since i have 640Mb ram, it seemed like a nice idea to load some frequently accessed apps in ram at boottime. One thing that bothered me however, is how i could easily move the files from a package to a ramdisk and let the system transparently load files from the ramdisk if they're available on the ramdisk and if they're not available, then load them from the harddisk.

First thing could easily be done with "equery files package", so you could easily make a startup script to copy all files from a selected set of packages on a ramdisk. (Or you could just use the script you wrote to find all the dependencies from an executable) The second thing, let the system transparently take files from the ramdisk if available there and otherwise from the harddisk seemed more difficult, till i read this. Back then however, this file system seemed only available for 2.4 kernels. Today, there are new versions and it should work on 2.6 kernels, and you can get it here. It's also available in portage btw. From what i remember when i read it, with unionfs it would be possible to make a filesystem that takes files from the ramdisk when they're available there and otherwise would access the harddisk. I intend to try to do something like this "some time in the future", but since you already made something similar, i thought you might be interested in unionfs...

ruben
Back to top
View user's profile Send private message
helmers
Guru
Guru


Joined: 16 Sep 2002
Posts: 548
Location: Oslo, Norway

PostPosted: Sun Apr 10, 2005 7:14 pm    Post subject: Great tip! Reply with quote

I haven't tried this (yet) but I think the idea is really nice, and if you could make it possible to do it via a script, so you could just preload say your 3-4 most used applications it would be great. The percieved speedup would be enormous.

Hope you (or someone else willing) will continue work on this!
_________________
C is for Cookies!
Back to top
View user's profile Send private message
s4kk3
Apprentice
Apprentice


Joined: 15 Oct 2004
Posts: 232
Location: Finland

PostPosted: Tue Apr 12, 2005 9:06 am    Post subject: Reply with quote

Whoa! This really speeds up things..

I did this sligthly different way.. I didn't do those initrd and linuxrc stuff and commented out tarring / in update-balls script but so far it looks like it works.. This takes over 80% of ram so playing games can be really pain
Back to top
View user's profile Send private message
Monstros
Tux's lil' helper
Tux's lil' helper


Joined: 07 Jul 2004
Posts: 111

PostPosted: Tue Apr 12, 2005 5:24 pm    Post subject: Reply with quote

i seems cool, but there's somthing I cannot understand, as the nioub I am.

If you put your lib in the ramdisk, ok, it's faster to load them, given that there're already loaded. But if you emerge something that update this lib. this lib will be updated and installed in the ramdisk, and when you put your comp off, the update is lost. I guess there's not this problem, cause a lot of peeps feels good with this ramdisk. But I just cannot understand how it works...


edit : another question : when the program need the lib, will it load it a second time in RAM (in the real ram, if I can say that), or will it use the in-ramdisk lib directly ?
_________________
Monstros Velu - Nioub
- Core 2 Duo E6600, eVGA n680i, 2Go DDR2 PC2-8500, 8800GTS 640Mo, 2x320Go SATA HD
- Fujitsu-Siemens M3438G 75005, Pentium M 750, 1Go DDR2, 2x80Go HD, 6800GO 256Mo, 17" 1440x900
Back to top
View user's profile Send private message
5a\/ag3
Apprentice
Apprentice


Joined: 08 Aug 2003
Posts: 175
Location: Spruce Grove Alberta, Canada

PostPosted: Mon May 16, 2005 3:36 am    Post subject: Reply with quote

Just found this. However Very cool howto! I am now configuring this with my baby devel :twisted:. should go good with 2Gb RAM and Dual Xeons.
_________________
Yeah... I got nothin for this field!
Back to top
View user's profile Send private message
chrissou
Guru
Guru


Joined: 22 Mar 2004
Posts: 473

PostPosted: Tue Jun 14, 2005 8:12 pm    Post subject: Reply with quote

hello i'm trying to use this howto to boost start of my application, but i have an error when i try to load linurx :

error line 22 /dev/console not found

I think i have an error with my root partition, i don't now how to work mknod utilities and i must mknod my sda drive ...
i try use mknod /mnt/initrd/dev/sda -b

Thanks a lot

excuze me for my very bad english
_________________
MacBookPro 17, OSX 10.6

Mon site oueb : http://www.generationdomotique.com
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Wed Jun 22, 2005 6:43 am    Post subject: Re: Alternative Method - Select Apps Reply with quote

unseen-enigma wrote:
So I ported over a basic chroot build script (takes core libs and ldd of a given app stores in a directory). I loaded openoffice(word-like part only), firefox, xorg, xine, and several (about 20) core kde applications into the /ramdisk I made under /ramdisk/bin and /ramdisk/lib (added to path).


hiya,

i wasn't sure if you were still keeping tabs on this thread, but if so, i was wondering where you got this build script at, or if you still have it handy. the reason i ask is because i like the sound of your method, but copying libs by hand doesn't sound like a fun thing to do on a Saturday night. ;)

also, how well does it handle symlinks? (i noticed that some "libraries" are actually symlinks to other versions.)
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
guitarman
Guru
Guru


Joined: 26 Apr 2004
Posts: 307
Location: http://mattsbox.dyndns.org:81/

PostPosted: Wed Jun 29, 2005 6:29 am    Post subject: Reply with quote

Is there any possable way to do this on a system with only one partition for / ? Only reason is I have no way to back everything up as I used > 100 gigs of my hd and I dont have a second hard drive big enouogh to back it up so I could repartition my disk.
Back to top
View user's profile Send private message
fE_rdy
Tux's lil' helper
Tux's lil' helper


Joined: 14 Feb 2003
Posts: 75
Location: Old Europe (m'kay Rummy?)

PostPosted: Tue Jul 12, 2005 12:08 am    Post subject: Re: HOWTO: Mount / in RAM and load apps instantly Reply with quote

Okay, your howto seems to be very cool. If it wouldnt be so late now, i just would make it work right now. :-) I just had a look in /usr/lib and there are more than 700 MB Ram in there. So an switching mechanism to add certain apps (and their corresponding libs) to the preload-cache is preferable. How could you get to know against which libs a given executable is linked? I thought ldd would do the job (for dynamic linked stuff). This doesn't work for firefox. Well, I go to bed now- but this thing will be far away from getting lost by sleep.

Well done!

yours,
f3rdy.
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Tue Jul 12, 2005 12:23 am    Post subject: Reply with quote

ldd will work with firefox... you have to find the executable (i think what you find in /usr/bin is a script that points to a script that points to a script which points to firefox).

you can also ldd the libraries themselves, as they depend on other libraries. it's not required unless you are doing a full chroot, but if you have the space for more libs and the time to copy them over, why not.

you could also unmerge a lot of packages you don't need (and --depclean). i did that and shaved off about 100MB from my /usr/lib

i'm ready to actualy try this out. i just haven't had the time to actually implement it.
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
0000000000
n00b
n00b


Joined: 14 Jul 2005
Posts: 3

PostPosted: Thu Jul 14, 2005 4:08 am    Post subject: Reply with quote

Yea I’m with you Sheepdogj15; does anyone have a cool build script to do this?

I also don't understand what would happen if I had firefox and all its libraries loaded into RAM and then ran emerge -u mozilla-firefox. Does this mean that it would just update the ram and get forgotten upon reboot, or would portage rewrite the directories to their original HD locations? If the RAM copy gets changed, could one just re-tar the RAM dir and save it to the HD before shutdown/reboot to keep changes?

Anyway selective loading of apps to RAM seems to be a very cool idea, I’m just unclear as to how system maintenance works...
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Thu Jul 14, 2005 4:42 am    Post subject: Reply with quote

i'm not sure how taht is addressed in the OP, but i but i have an idea.

the idea is that stuff will be loaded into RAM from a basic tarball. so, what do we do? we have two options: (A) create the tarball again everytime you do an emerge job, or (B) have it set to re-create the tarball before a shutdown-reboot. (i.e., add a tar command to local.stop). i'd actually do both, to be honest, using different names so you'd always have a backup of one gets corrupted.
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
BitJam
Advocate
Advocate


Joined: 12 Aug 2003
Posts: 2455
Location: Silver City, NM

PostPosted: Thu Jul 14, 2005 7:34 am    Post subject: Reply with quote

Monstros raises an interesting question. He asked if libraries could be run directory from the ram-disk or do they need to be loaded into "real" ram to run. Of course, they need to be loaded into "real" ram.

I really like the idea of getting programs to load fast, but Monstros' question makes me wonder if maybe a better way to get the speed-up (especially for people like me who have close to 1 gig in /usr/lib) is to just pre-load certain libraries into "real" ram at boot time.
Back to top
View user's profile Send private message
0000000000
n00b
n00b


Joined: 14 Jul 2005
Posts: 3

PostPosted: Thu Jul 14, 2005 6:13 pm    Post subject: Reply with quote

From the bigslide:
Quote:
###########Aside##########
If you just want to load certain applications from a RAM disk, you can do something like the following
Code:
##do this in advance
tar cpf /root/preload.tar /usr/bin/firefox /lib/and /lib/all /usr/lib/of /usr/lib/the /lib/raries/ it's/dependent /lib/on
##replace all the original bins and libraries with links to /preload/whatever
##Then put this in /etc/conf.d/local.start
mount -t tmpfs -o size=128m none /preload > /dev/null 2>&1
cd /preload && tar xfp /root/preload.tar
#########################

I just wanted to try this so i did
Code:
tar cpf /root/preload.tar /usr/bin/firefox /lib/ /usr/lib/MozillaFirefox/

and i have a nice 53M tar file, great. When i run
Code:
mount -t tmpfs -o size=128m none /preload > /dev/null 2>&1
i get no error, but then i cannot
Code:
cd /preload
...the dir isn't there. Am i retarded? Do i need something else in my fstab? ...currently i have the line
Code:
none         /dev/shm   tmpfs      defaults      0 0
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Thu Jul 14, 2005 6:40 pm    Post subject: Reply with quote

the thing is, this is my understanding based on my limited knowledge and a bit of time googling: if a library isn't already loaded into RAM (i.e., is being used by another application) the only time it is loaded is when it is needed by an app that is about to run. at least, i think that is so.

but, i have an idea, which at least in theory could work. basically, create a dummy application, and convince ld (if at compile time) or ld.so (every time the dummy app is called for) that it requires all of the libraries we want to mount into RAM. have the dummy run at boot time, and viola, you have tens or hundreds of MBs of libs in RAM. since the principle behind shared libraries is that if an app needs a shared library (basically anything with a .so on it) and that library is already in RAM, it will use that copy instead of loading it into RAM again. thus, we trick the system into autoloading libs into memory without using a ramdisk or tmpfs.

dummy would also have to stay in memory, or else the libs will be cached out (maybe set it up as a daemon that does nothing? :wink: )

there are a few problems i foresee, though. first off, i don't know know symbolic links are treated. for instance, if there's a foo.so.1.0.2, which is a symlink to foo.so.1.0.1, which is a symlink to foo.so.1, and all three are called by separate applications, whill it just load .1 into memory and trick the apps that need the other libs into using that, or will have have three libraries sitting in RAM, basically the same thing but with different names? i don't honestly know how the system handles that. my concern would be that if they are treated as distinct libraries and thus you get separate copies in RAM, that could theoretically be more "expensive" in terms of RAM than other methods listed here. (it also doesn't preload the applications that use those libs, but [1.] smaller binaries by themselves load into RAM pretty fast, and [2.] we can address that by mounting /usr/bin or /bin into RAM with tmpfs. my /usr/bin is only 80MB... my understanding is that it has been /usr/lib that has been a significant problem for many).

second and most important, this seems like it would be a bit unweildy to keep up do date. if you update to Firefox 1.1 in the near future, and (for example) 1.0.4 needed bar.so.1, and 1.1 needs bar.so.2, where they are totally different libs (meaning, one isn't just a symlink to the other), we would want a way to easily update our dummy app so we can get the new lib loaded (and maybe, get the old lib out of there if we don't need it anymore). we would really want an automated process to do it, because if you have to spend a bunch of time monkeying with this stuff everytime you emerge -uDN world, you'd end up spending more time than you saved.

of course, don't ask me how to make that dummy app in the first place. honestly? i don't know, but maybe someone else here does.
_________________
Sheepdog
Why Risk It? | Samba Howto


Last edited by Sheepdogj15 on Thu Jul 14, 2005 6:52 pm; edited 2 times in total
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Thu Jul 14, 2005 6:44 pm    Post subject: Reply with quote

0000000000 wrote:
i get no error, but then i cannot
Code:
cd /preload
...the dir isn't there. Am i retarded? Do i need something else in my fstab?


did you create the /preload directory in the first place? you can't mount a drive (real or pseudo) if there is nothing to mount it to.

otherwise, i dunno. i mount /tmp and /var/tmp into RAM and it works fine. try running that command in a console (mount -t tmpfs -o size=128m none /preload > /dev/null 2>&1) and see what errors you get.

BTW< i'd recommend a larger values for "size=". tmpfs will use as little RAM as it can in the first place, but IMO it isn't too hard to fill 128MB especially if we are talking about libraries and such)
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
BitJam
Advocate
Advocate


Joined: 12 Aug 2003
Posts: 2455
Location: Silver City, NM

PostPosted: Thu Jul 14, 2005 8:24 pm    Post subject: Reply with quote

Sheepdogj15 wrote:

second and most important, this seems like it would be a bit unweildy to keep up do date. if you update to Firefox 1.1 in the near future, and (for example) 1.0.4 needed bar.so.1, and 1.1 needs bar.so.2, where they are totally different libs (meaning, one isn't just a symlink to the other), we would want a way to easily update our dummy app so we can get the new lib loaded (and maybe, get the old lib out of there if we don't need it anymore). we would really want an automated process to do it, because if you have to spend a bunch of time monkeying with this stuff everytime you emerge -uDN world, you'd end up spending more time than you saved.

Perhaps the "ldd" command could be used to automate the selection of which libraries to load when an app gets updated.
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Thu Jul 14, 2005 8:45 pm    Post subject: Reply with quote

BitJam wrote:

Perhaps the "ldd" command could be used to automate the selection of which libraries to load when an app gets updated.


yeah, i'm thinking ldd would be used a lot for this kind of project.

maybe we should start a new thread (Unsupported Software or somewhere?) where we can get more people talking about how we could create this. i'd love to do this myself just for the learning experience, but i'm already swamped as it is, and anyways i'm assuming there isn't an easier way than what i proposed about (maybe there is and someone else will know it :?)
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
0000000000
n00b
n00b


Joined: 14 Jul 2005
Posts: 3

PostPosted: Thu Jul 14, 2005 9:00 pm    Post subject: Reply with quote

o thanks sheepdog, i am retarded, didn't have a /preload dir. Now it seems that firefox takes the same amount of time to load as when i loaded it successively before, ~3 seconds. It would usually take ~10 sec to start cold. Is this the best my machine will do (1ghz p3, 576 megs ram, 7200rmp hd), or does it theoretically help to replace the original files with symlinks to /preload?
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Thu Jul 14, 2005 9:07 pm    Post subject: Reply with quote

0000000000 wrote:
o thanks sheepdog, i am retarded, didn't have a /preload dir. Now it seems that firefox takes the same amount of time to load as when i loaded it successively before, ~3 seconds. It would usually take ~10 sec to start cold. Is this the best my machine will do (1ghz p3, 576 megs ram, 7200rmp hd), or does it theoretically help to replace the original files with symlinks to /preload?


yeah... though conceivably could just add /preload to your LD_PATH (don't ask, because i don't know how). you will want to go into /usr/lib, etc., find the libraries, rename them (incase something goes wrong, you can undo the process by naming them back) maybe add ".old" to the end of each name or something. then create symlinks that point to the libs in your /preload directory.
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
Sheepdogj15
Guru
Guru


Joined: 07 Jan 2005
Posts: 430
Location: Backyard

PostPosted: Sun Jul 31, 2005 1:16 am    Post subject: Reply with quote

has anyone heard about the Gigabyte i-RAM? i'm wondering if it will be supported by existing linux drivers (i don't see why not... just uses a SATA hdd interface), but if so, it could very well make these issues irrelevant. that is, assuming you have the money for it and 4 sticks of RAM, of course. ;)
_________________
Sheepdog
Why Risk It? | Samba Howto
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 3 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum