Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Filesystem for /usr/portage
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
vh4x0r
n00b
n00b


Joined: 11 May 2009
Posts: 37

PostPosted: Sun May 17, 2009 4:27 pm    Post subject: Filesystem for /usr/portage Reply with quote

Hi, I'm installing a new gentoo system on my laptop :) , and need some help with choosing a filesystem for /usr/portage. Should I choose reiser4 or ext4, or maybe ext2 will be faster? :!: Also can somebody help with the mount options?
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Sun May 17, 2009 4:35 pm    Post subject: Reply with quote

Ext3 or reiserFs4, ext4 is still experimental.
Back to top
View user's profile Send private message
vh4x0r
n00b
n00b


Joined: 11 May 2009
Posts: 37

PostPosted: Sun May 17, 2009 4:50 pm    Post subject: Reply with quote

well, which is gonna be faster, ext3 or reiserfs?
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Sun May 17, 2009 5:09 pm    Post subject: Reply with quote

ReiserFS, because /usr/portage have a lot of small files.
Back to top
View user's profile Send private message
yngwin
Retired Dev
Retired Dev


Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China

PostPosted: Sun May 17, 2009 7:04 pm    Post subject: Reply with quote

I recommend Reiser4.
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF
Back to top
View user's profile Send private message
mandas
n00b
n00b


Joined: 13 Nov 2006
Posts: 20

PostPosted: Sun May 17, 2009 7:27 pm    Post subject: Reply with quote

Check out this, it might help.

http://www.nabble.com/Recommendation-about-faster-(not-smaller)-filesystem-and-blocksize-combination-for-portage-tree-td22781217.html

D.

PS. I recommend ReiserFS
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Mon May 18, 2009 2:42 am    Post subject: Reply with quote

With small files and I mean a lot, ReiserFS is the best FS out there I think.
Back to top
View user's profile Send private message
Ormaaj
Guru
Guru


Joined: 28 Jan 2008
Posts: 319

PostPosted: Mon May 18, 2009 6:11 am    Post subject: Reply with quote

d2_racing wrote:
Ext3 or reiserFs4, ext4 is still experimental.
ext4 is considered stable - moreso than reiser4 (and faster). If you look at the benchmarks reiser4 pretty consistently bombs in comparison to everything else.

Heres just one example thats getting kind of old at this point. http://www.phoronix.com/scan.php?page=article&item=ext4_benchmarks&num=2

Also if you are concerned about file fragmentation and performance, isolating /var is a good idea, especially if you use ccache. It also contains a lot of other temporary files and caches, the portage installed package database, the emerge sandbox, and browser caches. That will probably be more critical than /usr/portage. Isolating /usr/portage might speed up syncs, but thats about it.
Back to top
View user's profile Send private message
tranquilcool
Veteran
Veteran


Joined: 25 Mar 2005
Posts: 1179

PostPosted: Mon May 18, 2009 8:55 am    Post subject: Reply with quote

played around with btrfs+compression for some but i have just
gone back to reiser4+compression. no benchmark but i had the
impression performance on btrfs didn't equal reiser4 after a while.
_________________
this is a strange strange world.
Back to top
View user's profile Send private message
gentoo-dev
Apprentice
Apprentice


Joined: 24 Jan 2006
Posts: 172

PostPosted: Mon May 18, 2009 10:49 am    Post subject: Reply with quote

ext2 with 1K blocks. Simple, clean, fast.
Back to top
View user's profile Send private message
djinnZ
Advocate
Advocate


Joined: 02 Nov 2006
Posts: 4831
Location: somewhere in L.O.S.

PostPosted: Mon May 18, 2009 1:06 pm    Post subject: Reply with quote

squashfs on xfs, reiser 3 is stable but suffer of fragmentation issues for portage (or you can format the portage partition at slowdon).
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt:wink:
mala tempora currunt...mater stultorum semper pregna est :evil:
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist :wink:


Last edited by djinnZ on Mon May 25, 2009 1:32 pm; edited 1 time in total
Back to top
View user's profile Send private message
yngwin
Retired Dev
Retired Dev


Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China

PostPosted: Tue May 19, 2009 9:27 pm    Post subject: Reply with quote

Ormaaj wrote:
d2_racing wrote:
Ext3 or reiserFs4, ext4 is still experimental.
ext4 is considered stable - moreso than reiser4 (and faster). If you look at the benchmarks reiser4 pretty consistently bombs in comparison to everything else.

Says who?
Ext4 considered stable??? That's ridiculous. Reiser4 has been around much, much longer and is a lot more stable. So if you want stable and fast, Reiser4 is the way to go. But you're free to use whatever else. :wink:
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF
Back to top
View user's profile Send private message
yngwin
Retired Dev
Retired Dev


Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China

PostPosted: Tue May 19, 2009 9:29 pm    Post subject: Reply with quote

djinnZ wrote:
squashfs or xfs

XFS for portage doesnt make sense. XFS's strength is with big files.
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF
Back to top
View user's profile Send private message
i92guboj
Bodhisattva
Bodhisattva


Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)

PostPosted: Tue May 19, 2009 10:08 pm    Post subject: Reply with quote

For me it'd be ext2, with 1k blocks, yes.

There's nothing that justifies the extra cpu load of a journal on a fs where there's nothing really valuable that you can't get back with a sync. But this much depends on your machine. I don't usually have available last generation hardware, and cpu is a valuable resource for me, so I try to save every cpu cycle that I can.

I really laugh when people seem that worried about the speed of the portage partition. It's a nonsense taking into account that compilations can last hours. Who cares if a search operation takes 4 more seconds?
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Wed May 20, 2009 10:48 am    Post subject: Reply with quote

I use ReiserFS for /usr/portage, /usr/src, /var/tmp/ccache, /var/tmp/portage.

It's great for tons of small files, much better than XFS (which is what I use for everything else).

For searching a package I just use eix.

ext2 with small blocks sounds good too, but you have to be careful with extX in general... you run out of inodes, it's over for you, only solution (apart from deleting files) is reformatting this time with more inodes. There's someone every day in the #gentoo channel with exactly this kind of problem.
Back to top
View user's profile Send private message
djinnZ
Advocate
Advocate


Joined: 02 Nov 2006
Posts: 4831
Location: somewhere in L.O.S.

PostPosted: Wed May 20, 2009 12:47 pm    Post subject: Reply with quote

yngwin wrote:
XFS's strength is with big files
I agree but the problem with reiser 3 (an instable filesystem can be reasonable for testing never to work IMHO) and portage is the fragmentation and xfs not suffer about this; is the better choiche if you not use a separate partition and a reasonable alternative if you have a large amount of files in portage/distfiles.
In the computers of mine I use squashfs for portrage over an xfs partition with all the downloaded packages and the local overlay and all work fine.
make.conf:
PORTDIR_OVERLAY="/usr/portage/local"
PORTDIR=/usr/portage/gentoo
PKGDIR=/usr/portage/pkg/packages
DISTDIR=/usr/portage/pkg/distfiles
/usr/portage is on a xfs partition
xfs_info /usr/portage:
meta-data=/dev/sdd5              isize=256    agcount=64, agsize=168871 blks
         =                       sectsz=512   attr=1
data     =                       bsize=512    blocks=10807712, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=512    ascii-ci=0
log      =internal               bsize=512    blocks=2048, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=65536  blocks=0, rtextents=0
and /usr/portage/gentoo is squashfs.
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt:wink:
mala tempora currunt...mater stultorum semper pregna est :evil:
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist :wink:
Back to top
View user's profile Send private message
coolsnowmen
Veteran
Veteran


Joined: 30 Jun 2004
Posts: 1479
Location: No.VA

PostPosted: Wed May 20, 2009 6:00 pm    Post subject: Reply with quote

I have had reiserfs for /usr/portage for about 9 months now. And every couple weeks I get strange I/O errors on a sync. This forces me to unmount it, fsck it, and remount it, then sync again. I don't have any other problems with the other filesystems on the disk (ext4 and ext2).

And as far as the "lots of small files" argument. It doesn't really hold true because /usr/portage doesn't have lots of files per directory. It only has a couple small files, so I don't see any massive speed boost you would get vs. ext2 that would make it worth the problems I have with it.

So, by my experience, I would not recommend it.
_________________
emerge: there are no ebuilds to satisfy "moo"
Back to top
View user's profile Send private message
DONAHUE
Watchman
Watchman


Joined: 09 Dec 2006
Posts: 7651
Location: Goose Creek SC

PostPosted: Wed May 20, 2009 6:36 pm    Post subject: Reply with quote

2 reiserfs /usr/portage each has ~139896 items, totalling ~3.6 GB and neither has had an error on a sync in 3 years of life.
mileage varies.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Wed May 20, 2009 6:45 pm    Post subject: Reply with quote

coolsnowmen wrote:
So, by my experience, I would not recommend it.


I don't trust reiserfs either, that's why I only use it for portage... since it checksums every file, corruption on that partition does not matter.

But so far, no issues.
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10587
Location: Somewhere over Atlanta, Georgia

PostPosted: Wed May 20, 2009 8:00 pm    Post subject: Reply with quote

coolsnowmen wrote:
...And as far as the "lots of small files" argument. It doesn't really hold true because /usr/portage doesn't have lots of files per directory. ...
I believe that a directory with only a couple of files in it is itself a "small file", which is why I believe that the argument does hold true.

For what it's worth, I've been using ReiserFS on /usr/portage on several machines for years now and it's been rock solid for me.

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
coolsnowmen
Veteran
Veteran


Joined: 30 Jun 2004
Posts: 1479
Location: No.VA

PostPosted: Wed May 20, 2009 8:30 pm    Post subject: Reply with quote

john_r_graham wrote:
coolsnowmen wrote:
...And as far as the "lots of small files" argument. It doesn't really hold true because /usr/portage doesn't have lots of files per directory. ...
I believe that a directory with only a couple of files in it is itself a "small file", which is why I believe that the argument does hold true.

My mistake, I went back and read the kernel notes on resiserfs, it says:
Quote:
ReiserFS is as fast as ext2, but is very efficient with large directories and small files.

I remembered it wrong as "large directories of small files" but I guess either satisfy that condition.
_________________
emerge: there are no ebuilds to satisfy "moo"
Back to top
View user's profile Send private message
pigeon768
l33t
l33t


Joined: 02 Jan 2006
Posts: 683

PostPosted: Thu May 21, 2009 10:20 pm    Post subject: Reply with quote

djinnZ wrote:
yngwin wrote:
XFS's strength is with big files
I agree but the problem with reiser 3 (an instable filesystem can be reasonable for testing never to work IMHO) and portage is the fragmentation and xfs not suffer about this; is the better choiche if you not use a separate partition and a reasonable alternative if you have a large amount of files in portage/distfiles.
There's going to be effectively zero fragmentation in /usr/portage. Ebuilds are generally only a block or two.

xfs is an absolutely miserable filesystem choice for /usr/portage and /var/tmp/portage. xfs is very good at modifying enormous files. If you're running a huge database, put your database on a separate xfs partition - performance will be great. However, there's simply too much overhead involved for it to be useful at parsing through and opening each of the enormous number of files in /usr/portage, or with creating, deleting, or modifying a bunch of files during building.

IMO it's not even very good at running around through /usr/bin.

reiserfs is a great choice for /usr/portage and /var/tmp/portage. Even if you don't like reiserfs because you think it's unstable, (you're wrong) the data in /usr/portage and /var/tmp/portage is trivial to regenerate, and there are hefty speed gains by using it instead of xfs for instance.
Back to top
View user's profile Send private message
djinnZ
Advocate
Advocate


Joined: 02 Nov 2006
Posts: 4831
Location: somewhere in L.O.S.

PostPosted: Sat May 23, 2009 10:16 pm    Post subject: Reply with quote

reiserfs 4 is unstable (reported usable but not stable), this is a fact, reiserfs 3.6 is really stable (more than ext3 in my experience) and the better choiche for /var/tmp IMHO (and I use it, not only xfs on the computers of mine).
You speak about reiserfs but what is the version?

The xfs partition on the pc of mine host directly only the ebuilds written by me, all the files are ever bigger than 10 MB and are only tar archives and squashfs images. All the ebuilds are on squashfs not on xfs.
And the content of /usr/portage/distfiles can be downloaded (but some are really painful as the damned multiple java versions from 1.3 to 1.6 than i need) but is a large amount of data so i will be sure (and never will use an unstable filesystem) and reiser is not the better choiche for this.

In the case you have two separate partitions, one for /usr/portage and one for /usr/portage/distfiles, the fragmentation and the performance degrade will not be a problem.

uh, i have forget to say than i use to sync from mirrors on a separate directory (hosted on reiserfs) in the server and later I update the portage on the other computers, in order to prevent mistakes (and copy a file directly is more useful than update it via rsync in a local lan) and errors (the others pcs are not able to sync).
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt:wink:
mala tempora currunt...mater stultorum semper pregna est :evil:
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist :wink:
Back to top
View user's profile Send private message
BitJam
Advocate
Advocate


Joined: 12 Aug 2003
Posts: 2508
Location: Silver City, NM

PostPosted: Sun May 24, 2009 9:51 pm    Post subject: Reply with quote

I've tried several different file systems for /usr/portage over the years. I'm currently using XFS with the smallest blocksize possible. I can typically sync in well under two minutes. If I sync regularly, the time goes down to about one minute. IMO the small block size outweighs almost all other factors because it reduces the disk I/O (due to all the tiny files). My /usr/portage partition is using 346M according to df -h. I suggest using the FS that has the smallest blocksize. But don't use Reiser-3. It does not fragment gracefully and quickly slows down to a crawl.

If syncing starts to slow down then I copy, erase, and repopulate which works to de-frag. I do this maybe one a year or so.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Mon May 25, 2009 9:16 am    Post subject: Reply with quote

BitJam wrote:
If syncing starts to slow down then I copy, erase, and repopulate which works to de-frag. I do this maybe one a year or so.


If you're using XFS you could just use the defrag tool that comes with XFS...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum