| View previous topic :: View next topic |
| Author |
Message |
fangorn
Veteran
Joined: 31 Jul 2004
Posts: 1886
|
 Posted: Mon May 25, 2009 9:59 am Post subject: |
 |
|
I prefer reiserfs (v3) for size reasons. (support vor 512 byte blocks)
Makes it small enough to place the whole filesystem in RAM if you need to. -> flying cows! 
_________________
Video Encoding scripts collection | Project page |
|
| Back to top |
|
 |
gentoo-dev
Apprentice
Joined: 24 Jan 2006
Posts: 172
|
| Posted: Mon May 25, 2009 10:38 am Post subject: |
|
|
| BitJam wrote: | | I've tried several different file systems for /usr/portage over the years. I'm currently using XFS with the smallest blocksize possible. I can typically sync in well under two minutes. If I sync regularly, the time goes down to about one minute. |
And you're happy with that? I emerge --sync once in a while and it takes 20-30s. I use a local rsync mirror, so network speed is not an issue in my case. |
|
| Back to top |
|
|
i92guboj
Bodhisattva
Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)
|
| Posted: Mon May 25, 2009 10:52 am Post subject: |
|
|
| gentoo-dev wrote: | | BitJam wrote: | | I've tried several different file systems for /usr/portage over the years. I'm currently using XFS with the smallest blocksize possible. I can typically sync in well under two minutes. If I sync regularly, the time goes down to about one minute. |
And you're happy with that? I emerge --sync once in a while and it takes 20-30s. I use a local rsync mirror, so network speed is not an issue in my case. |
Which is irrelevant to the thread and is even more irrelevant since cron exists. It doesn't really matter if it takes 30s, 2mins or 1 hour, just add the job to cron.daily or whichever one you prefer. If there's a task that can run unattended, that's emerge --sync  |
|
| Back to top |
|
|
djinnZ
Advocate
Joined: 02 Nov 2006
Posts: 4831
Location: somewhere in L.O.S.
|
| Posted: Mon May 25, 2009 1:32 pm Post subject: |
|
|
I agree, the problem with reiser 3 and portage is the performance decrease, but not for the sync operation, at least for the metadata rebuild (called at end of the sync).
And the fragmentation is not determined by the large number of small files (reiser 3 is optimized to host this), is determined by the precence of a large number of small files (<1kb) and a large number of "normal" (10->250 MB is not really a very big filesize) files.
Try to put the source tars and the binaries on another partition and you will not see any slowdown.
And not forget the filesystem than host the metadata cache.
@BitJam : Using a small blocksize will only spare space on disk, never increasing the performance. There is a reason for having a default blocksize of 4kb for every filesystem.
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt
mala tempora currunt...mater stultorum semper pregna est 
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist |
|
| Back to top |
|
|
BitJam
Advocate
Joined: 12 Aug 2003
Posts: 2508
Location: Silver City, NM
|
| Posted: Mon May 25, 2009 5:53 pm Post subject: |
|
|
| frostschutz wrote: | | If you're using XFS you could just use the defrag tool that comes with XFS... |
Thanks!
Although I wonder if that tool works as well as brute-force copying. When the portage partition slows down, I don't think the problem is *file* fragmentation since most of the files are smaller than a single block. I think the slowdown is caused when files within a directory become scattered on the disk. Assume each file is contained in one block. Then defragging files gains you nada. But what you do want is that the order of the blocks on the disk is the same as the order in which the files will be read. I believe the copying hack provides this ordering assuming the copying occurs in the same order that Portage uses for searching.
Of course, all of this is rather crazy because it is crazy to try to use any file system as if it were a database. IMO this is a fundamental architectural flaw in Portage. I understand the rationale for why it is done this way: the devs don't want Portage to depend on a database because that would defeat the bootstrap idea of Gentoo. They don't want the boards flooded with "I accidentally unmerged MySQL and now emerge won't work". So then the next suggestion is to have two systems. The current FS based system and also a DB based system that both deal with the same information. People would use the FS system to get started and then switch to the DB based system for better performance. I think there have been many attempts at this but they have never full succeeded because they are hampered by the fact that the current FS based system is "good enough" and no one really want to maintain two parallel systems. Therefore we're stuck with this craziness of pretending the FS is a database. There is a similar craziness in the init system that pretends Bash is a full-fledged high level language. But it's not so they have to keep playing tricks with increasing the size of the Bash environment which then temporarily breaks other programs (such as xargs). But, to paraphrase Kahlil Gibran: | Quote: | | People who do not forgive Gentoo her little faults will never enjoy her great virtues. |
Anyway, thanks for the tip regarding the XFS defrag tools. |
|
| Back to top |
|
|
kdorf
n00b
Joined: 28 May 2009
Posts: 2
|
| Posted: Fri May 29, 2009 3:30 pm Post subject: |
|
|
| yngwin wrote: | | Ext4 considered stable??? That's ridiculous. Reiser4 has been around much, much longer and is a lot more stable. So if you want stable and fast, Reiser4 is the way to go. But you're free to use whatever else. |
http://en.wikipedia.org/wiki/Ext4
I just started using Ext4 myself though. I don't tend to jump on the bandwagon of "it's marked stable now, so let's use it!" I've always waited for a few bugs to be ironed out. =) Have to say though, using it on my laptop I seem to get really good performance, though I don't have a separate partition for /usr/portage. I should do that. |
|
| Back to top |
|
|
i92guboj
Bodhisattva
Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)
|
| Posted: Fri May 29, 2009 3:53 pm Post subject: |
|
|
[offtopic]
| kdorf wrote: | | yngwin wrote: | | Ext4 considered stable??? That's ridiculous. Reiser4 has been around much, much longer and is a lot more stable. So if you want stable and fast, Reiser4 is the way to go. But you're free to use whatever else. |
http://en.wikipedia.org/wiki/Ext4
I just started using Ext4 myself though. I don't tend to jump on the bandwagon of "it's marked stable now, so let's use it!" I've always waited for a few bugs to be ironed out. =) Have to say though, using it on my laptop I seem to get really good performance, though I don't have a separate partition for /usr/portage. I should do that. |
This really depends on who do you ask to.
The "it's been around much longer" argument from yngwin is a double blade weapon. Windows has also been around longer than Linux around, following the same logic it must be a a lot more stable. Maybe it's not the fairest example because to start with, Linux is not even a complete OS, but you get the point. Ext4 has come a very long way in a much shorter time than reiser4, and even if there are bugs (and nasty ones indeed) you must also look at the other side of the coin and agree with me that reiser4 has had equally nasty bugs for a much longer time.
I am not here to argue what's better, it's been long since I studied things like these and anyway I lack the understanding about the reiser4 internals to be able to argue technically about it. There's already a lot of that on the net. I'll just leave it at "it depends on the lenses you look through". Personally, I'd invest my energies in defending something like btrfs, because it's the way to go for the future. In my eyes, reiser4 has little present, and neither of reiser* and ext* are modern fs's oriented to the future, which in my humble opinion lies on btrfs and zfs. Those two are the fs's that extend the fs paradigm into this new century. All the rest are basically a remake of the same old film of a monster chasing teenagers: the blonde girl survives, the black guy dies first (just telling the stereotype, not that I like these movies at all).
[/offtopic] |
|
| Back to top |
|
|
JohnBlbec
Guru
Joined: 08 Feb 2003
Posts: 306
|
| Posted: Sat May 30, 2009 4:03 pm Post subject: |
|
|
i used reiserfs, now i am using ext2 and i am really satisfied.
| Code: |
for /usr/portage ............. mke2fs -b 1024 -N 200000 /dev/xxx
for /usr/portage/distfiles ... mke2fs -b 4096 -T largefile /dev/yyy
|
|
|
| Back to top |
|
|
Captain Newbie
Apprentice
Joined: 22 Dec 2006
Posts: 182
Location: Socal
|
| Posted: Sun May 31, 2009 6:55 am Post subject: |
|
|
| yngwin wrote: | | Ormaaj wrote: | | d2_racing wrote: | | Ext3 or reiserFs4, ext4 is still experimental. |
ext4 is considered stable - moreso than reiser4 (and faster). If you look at the benchmarks reiser4 pretty consistently bombs in comparison to everything else. |
Says who?
Ext4 considered stable??? That's ridiculous. Reiser4 has been around much, much longer and is a lot more stable. So if you want stable and fast, Reiser4 is the way to go. But you're free to use whatever else. |
Only-sort-of-offtopic: If you want your bugs fixed...I can tell you which one is in the mainline kernel tree, and which one isn't. ReiserFS is not all it's cracked up to be and using Reiser4 requires you to run the -mm trees or pull from the -mm tree and write the changes over a mainline kernel. I think there are bad politics at work against reiserfs/reiser4... :/ It's lousy, but politics plays a big role in open source. If we could all stop fighting each other we would've knocked Microsoft clean out of business by now with the energy that has been expended on religious wars.
My Portage tree lives happily on ext2.
Ontopic: I agree with what's been said about btrfs...that is an exciting filesystem. However, the Extended filesystems aren't going anywhere, no matter how much we might throw rocks at them. Ext3 is really quite good and I'm excited about Ext4. Resier is still alright, although some of its features scare me when I consider that I might not be able to get my data off if I hose the tree...
_________________
/* Nobody will ever see this message  */
panic("Cannot initialize video hardware\n");
"As much as it pains me, we hope that developers know what they're doing." - wolf31o2 |
|
| Back to top |
|
|
yngwin
Retired Dev
Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China
|
| Posted: Sun May 31, 2009 10:46 am Post subject: |
|
|
| Captain Newbie wrote: | | [...] using Reiser4 requires you to run the -mm trees or pull from the -mm tree and write the changes over a mainline kernel. |
Not true. There are patches against vanilla linux releases.
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF |
|
| Back to top |
|
|
Captain Newbie
Apprentice
Joined: 22 Dec 2006
Posts: 182
Location: Socal
|
| Posted: Sun May 31, 2009 2:02 pm Post subject: |
|
|
| yngwin wrote: | | Captain Newbie wrote: | | [...] using Reiser4 requires you to run the -mm trees or pull from the -mm tree and write the changes over a mainline kernel. |
Not true. There are patches against vanilla linux releases. |
Very well, I stand corrected.
_________________
/* Nobody will ever see this message */
panic("Cannot initialize video hardware\n");
"As much as it pains me, we hope that developers know what they're doing." - wolf31o2 |
|
| Back to top |
|
|
|
Installing Gentoo
