Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
TIP: *really* speed up portage - at the expense of RAM
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
bobobo
Tux's lil' helper
Tux's lil' helper


Joined: 24 Nov 2005
Posts: 122

PostPosted: Tue Aug 15, 2006 3:55 pm    Post subject: TIP: *really* speed up portage - at the expense of RAM Reply with quote

EDIT:

removed, as it is not really worth the drawbacks.
Furthermore, as a more practical alternative, simply use paludis instead of portage.


Last edited by bobobo on Thu Aug 23, 2007 1:47 am; edited 1 time in total
Back to top
View user's profile Send private message
fangorn
Veteran
Veteran


Joined: 31 Jul 2004
Posts: 1886

PostPosted: Wed Aug 16, 2006 11:04 am    Post subject: Reply with quote

Nice!

I already use tmpfs for /var/tmp/portage, which speeds up compile pretty neat. I only use harddisk space when compiling monster packages like wxGTK or OOo. Dont have 5 or 6 GB RAM to waste :wink:

But this one could have great impact on searching or calculating dependencies.

Thanks.
_________________
Video Encoding scripts collection | Project page
Back to top
View user's profile Send private message
slick
Bodhisattva
Bodhisattva


Joined: 20 Apr 2003
Posts: 3490

PostPosted: Wed Aug 16, 2006 1:51 pm    Post subject: Reply with quote

In the german forum at this days we diskuss about a way like this, but using unionfs and not fully load /usr/portage in RAM, only the changes of an emerge --sync. It's proof of concept only and not ready to use it (and like here, but without squashfs)

Code:
mv /usr/portage /usr/portage_org
mkdir /usr/portage

mv /var/cache/edb/dep /var/cache/edb/dep_org
mkdir /var/cache/edb/dep

mkdir /cache
mount -t tmpfs tmpfs -o size=800M,nr_inodes=1M /cache
mkdir /cache/portage
mkdir /cache/dep

mount -t unionfs -o dirs=/cache/portage:/usr/portage_org unionfs /usr/portage
unionctl /usr/portage --mode /usr/portage_org ro

mount -t unionfs -o dirs=/cache/dep:/var/cache/edb/dep_org unionfs /var/cache/edb/dep
unionctl /var/cache/edb/dep --mode /var/cache/edb/dep_org ro

emerge --sync
Back to top
View user's profile Send private message
stahlsau
Guru
Guru


Joined: 09 Jan 2004
Posts: 565
Location: WildWestwoods

PostPosted: Wed Aug 16, 2006 3:14 pm    Post subject: Reply with quote

mmh...talking about fair people who "emerge --sync" not more than once a day, this is pretty much nonsense i think. I mean, type in "emerge --sync", browse the web or do some stuff, and after some time check back what packages need to get upgraded. Who cares if the syncing process takes a few seconds more or less?
And for searching the portage tree, "eix -sS" does a great job with instant delivery of it's findings.

So, what's the point in this other than wasting ram? ;)
_________________
Join the adopt an unanswered post initiative today
Back to top
View user's profile Send private message
bobobo
Tux's lil' helper
Tux's lil' helper


Joined: 24 Nov 2005
Posts: 122

PostPosted: Wed Aug 16, 2006 6:46 pm    Post subject: Reply with quote

Yes indeed, it's a huge waste of ram.
However, if you frantically emerge 100s of packages everyday, it's a huge improvement (granted that you have enough ram). The point is not really to speed up the syncing, which is rare like you said, but more the dependency checking and searching.
The portage system isnt designed to handle such a huge number of packages, so until the system changes (which i don't see happen anytime soon), this is a solution for those who like wasting their ram. More 'reasonable' approaches would be the squashfs/unionfs or jffs2 method. I tried jffs2 before, it seems to work well. There is also cdb (can be found on the wiki), but it doesnt seem to give a really big improvement, and breaks often with new portage releases.

I think that portage is fairly well suited for tmpfs, for these reasons :
- it is accessed often (well that depends of your usage of emerge), and reading the portage tree from the HD is painfully slow due to the number of files.
- it isn't modified often (only when syncing).
- if some screw up happens, it can easily be restored.

edit: and also, it is only truly useful if you have far more ram than you normally use (or if you simply don't use swap), otherwise you'll lose the benefits due to swapping.


Last edited by bobobo on Thu Aug 17, 2006 12:01 am; edited 2 times in total
Back to top
View user's profile Send private message
slick
Bodhisattva
Bodhisattva


Joined: 20 Apr 2003
Posts: 3490

PostPosted: Wed Aug 16, 2006 6:57 pm    Post subject: Reply with quote

slick wrote:
In the german forum at this days we diskuss about a way like this, but using unionfs and not fully load /usr/portage in RAM, only the changes of an emerge --sync.

I benchmark my idea and its not recommered, its really slower. Forget my idea.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum