Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How can I measure fragmentation on ReiserFS partition?
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
Enlight
Advocate
Advocate


Joined: 28 Oct 2004
Posts: 3514
Location: Alsace (France)

PostPosted: Fri Feb 03, 2006 8:45 am    Post subject: Reply with quote

There are actually no (reiser) tools to check reiserfs'fragmentation, and reiser can't check by itself. _droop- script is really interesting, but should maybe be extended as it only reports for a give base tree, how much files are fragmented on how much present files, but it won't tell you how bad files are fragmented. As an exemple, I've got a 12 Go backup (tar?bz2) on a reiser-3.6 partition, and file frag reports it to be fraged in 80 pieces.

Last, XFS doesn't defrag itself on the fly, it "just" handle perfectly (or something very close to perfection) external fragmentation and then takes much more avantages of readahead and hard disk cache. Plus XFS seems to be the fs to have on smp and/or raid machines.
_________________
le "lol" est aux boulets ce que le ";" est au programmeurs
Back to top
View user's profile Send private message
dhave
Apprentice
Apprentice


Joined: 28 Oct 2005
Posts: 298
Location: Still outside the Matrix ...

PostPosted: Fri Feb 03, 2006 9:16 am    Post subject: Reply with quote

feld wrote:
I too have switched primarily to XFS not too long ago. Performs great!
Right now I have an LVM that has portage, tmp, and /var/tmp on Reiserfs. Figured that would give me pretty decent speed where it may occasionally be needed... and all 3 of those can always be fully wiped without any consquences which can bring the speed back up to its original level.

I'm really interested in your approach and am considering something similar. Can you elaborate a bit? Or even do a simple how-to? So you think the speed difference in reiserfs and xfs warrants putting your most active files on reiserfs? Also, your post has got me looking into LVM. Sounds pretty cool.
_________________
This space available!
Back to top
View user's profile Send private message
_droop_
l33t
l33t


Joined: 30 May 2004
Posts: 957

PostPosted: Fri Feb 03, 2006 9:24 am    Post subject: Reply with quote

Re,

I have modified my script. Now it reports the percentage of fragmented files. It should be more useful...

It needs one argument to work : the path of the fs to analyse. It should work on all fs type... (but it seems that reiserfs is the only fs which has no tools to report fragmentation...).

Reguards.
Back to top
View user's profile Send private message
gvs
n00b
n00b


Joined: 29 May 2004
Posts: 45
Location: EU - Belgium

PostPosted: Fri Feb 03, 2006 9:31 am    Post subject: Reply with quote

_droop_ wrote:
Re,

I have modified my script. Now it reports the percentage of fragmented files. It should be more useful...

It needs one argument to work : the path of the fs to analyse. It should work on all fs type... (but it seems that reiserfs is the only fs which has no tools to report fragmentation...).

Reguards.


Where can I download this script? If the version in your previous post corrected?
_________________
Registered Linux user #249404 - September 1997
Back to top
View user's profile Send private message
_droop_
l33t
l33t


Joined: 30 May 2004
Posts: 957

PostPosted: Fri Feb 03, 2006 10:08 am    Post subject: Reply with quote

Yes,

I corrected the script on the previous page...

I scanning all my fs, to see if there are bugs... There are problems with some characters in filename. I will update the script to correct these errors.
Back to top
View user's profile Send private message
dhave
Apprentice
Apprentice


Joined: 28 Oct 2005
Posts: 298
Location: Still outside the Matrix ...

PostPosted: Fri Feb 03, 2006 11:10 am    Post subject: Reply with quote

_droop_ wrote:
Yes,

I corrected the script on the previous page...

I scanning all my fs, to see if there are bugs... There are problems with some characters in filename. I will update the script to correct these errors.

_droop_, Thanks for the script. If it's doing what it seems to be doing, then it's a nice little tool. I tried it on my /opt directory, and it gave a nice report, as follows:

Code:
# /usr/local/sbin/fragshow.sh /opt
1.90536678310575& non contiguous files, 1.05009526833916 average fragmentation.

Also, it was fairly speedy on my P3 850MHz, 512Mb RAM machine. I'll let it work on some bigger, probably more fragmented directories and let you know if I run into problems.

Thanks again.
_________________
This space available!
Back to top
View user's profile Send private message
dhave
Apprentice
Apprentice


Joined: 28 Oct 2005
Posts: 298
Location: Still outside the Matrix ...

PostPosted: Fri Feb 03, 2006 11:41 am    Post subject: Reply with quote

Hey, _droop_, I ran the script on my fairly large /home directory and got good results, apart from a slight filename-related problem. I sent you a PM about this.

One question about interpreting the results. I got this:

Code:
4.09904304935144% non contiguous files, 1.20072688861238 average fragmentation.

Can I assume that this means ~4.1% of my files are non-contiguous, and that *those non-contiguous files* are, on average, ~1.2% fragmented? Or does the "average fragmentation" figure mean something else.

Thanks again for what could be a very useful tool for all of us.
_________________
This space available!
Back to top
View user's profile Send private message
_droop_
l33t
l33t


Joined: 30 May 2004
Posts: 957

PostPosted: Fri Feb 03, 2006 1:03 pm    Post subject: Reply with quote

dhave wrote:

One question about interpreting the results. I got this:

Code:
4.09904304935144% non contiguous files, 1.20072688861238 average fragmentation.

Can I assume that this means ~4.1% of my files are non-contiguous, and that *those non-contiguous files* are, on average, ~1.2% fragmented? Or does the "average fragmentation" figure mean something else.


The 1.2 means that (in average) your files are in 1.2 pieces on the disk. Ideally, it should report "0% non contiguous files, 1 average fragments." (i changed the text, I hope it is better now).

For the speed, the script is quite slow, since it has to examine all regular files of the examinated fs. But, it will be hard to do it better...
Back to top
View user's profile Send private message
_droop_
l33t
l33t


Joined: 30 May 2004
Posts: 957

PostPosted: Fri Feb 03, 2006 6:12 pm    Post subject: Reply with quote

I have posted my script in the Documentation section : http://forums.gentoo.org/viewtopic-p-3081971.html
Back to top
View user's profile Send private message
feld
Guru
Guru


Joined: 29 Aug 2004
Posts: 593
Location: WI, USA

PostPosted: Fri Feb 03, 2006 6:42 pm    Post subject: Reply with quote

dhave wrote:
feld wrote:
I too have switched primarily to XFS not too long ago. Performs great!
Right now I have an LVM that has portage, tmp, and /var/tmp on Reiserfs. Figured that would give me pretty decent speed where it may occasionally be needed... and all 3 of those can always be fully wiped without any consquences which can bring the speed back up to its original level.

I'm really interested in your approach and am considering something similar. Can you elaborate a bit? Or even do a simple how-to? So you think the speed difference in reiserfs and xfs warrants putting your most active files on reiserfs? Also, your post has got me looking into LVM. Sounds pretty cool.



There are plenty of how-tos available. It is actually quite simple.

1. emerge lvm2

2. have Device Mapper support in your kernel (it's in "Multiple devices driver support (RAID and LVM)")

3. boot off a live cd

4. somehow redo your partitioning scheme, or if you have more drives you want to play with it on create a partition with 8e as the type. (I think its 8e... Linux LVM). Don't forget to have a handy stage4 backup. That's what I did, wiped off / after backing it up to my little server, then copied it back over and extracted after creating and mounting everything.

5. Now you'll have to use the vgcreate command to create a volume group. you will tell it what LVM partitions on what hard drives you want to be a part of it. We'll use the variable $VOLUMEGROUP to cover it's name in this mini-howto.

6. Now use lvcreate to create logical volumes out of that volume group you created. You can specify by size or by the # of the LVM Physical Extents (PE). The PE are usually 4.00 MB each. We'll call these logical volumes $LOGICAL
. okay, now that you have your volumes created you can put filesystems on them.

fsck.WHATEVER /dev/$VOLUMEGROUP/$LOGICAL

7. now just put it in your fstab as the device being /dev/$VOLUMEGROUP/$LOGICAL and the correlating mountpoints that you desire.


That's basically it. The only thing I didn't mention is that if you boot from the livecd and want to mount volumes you have already created you have to activate them first. Something like vgchange -a y $VOLUMEGROUP does the trick.


Did I mention that with some filesystems you can EXTEND the logical volume's size and then extend the filesystem itself while it is still mounted? I think Reiserfs and XFS can do this. Remember, XFS only grows -- you cannot shrink it.


My situation:

I have a $VOLUMEGROUP named vg and 3 $LOGICALs named tmp, tmpportage, and portage. They are 5GB, 10GB, and 5GB in size and are mounted to /tmp, /var/tmp, and /usr/portage.



Yes it does seem to be a very fast and efficient solution.



-Feld
_________________
< bmg505> I think the first line in reiserfsck is

if (random(65535)< 65500) { hose(partition); for (i=0;i<100000000;i++) print_crap(); }
Back to top
View user's profile Send private message
BlackEdder
Advocate
Advocate


Joined: 26 Apr 2004
Posts: 2588
Location: Dutch enclave in Egham, UK

PostPosted: Fri Feb 03, 2006 6:51 pm    Post subject: Reply with quote

See:
http://www.gentoo.org/doc/en/lvm2.xml
for a more indepth guide.
Back to top
View user's profile Send private message
vputz
Guru
Guru


Joined: 16 Mar 2005
Posts: 308
Location: Oxford, England

PostPosted: Fri Feb 03, 2006 10:28 pm    Post subject: Reply with quote

...and to append to that and vector this thread even further from its intended purpose--if you're playing with LVM, I'd actually consider just jumping straight to EVMS as the front-end for it. I've got it running on two boxes so far, and don't see any disadvantages yet (and you can play some really interesting games with it...).
Back to top
View user's profile Send private message
sn4ip3r
Guru
Guru


Joined: 14 Dec 2002
Posts: 325
Location: Tallinn, Estonia

PostPosted: Sat Feb 04, 2006 3:36 pm    Post subject: Reply with quote

Could anyone test this "defragmenter" and see if it reduces the defragmentation?
Back to top
View user's profile Send private message
dhave
Apprentice
Apprentice


Joined: 28 Oct 2005
Posts: 298
Location: Still outside the Matrix ...

PostPosted: Sat Feb 04, 2006 4:24 pm    Post subject: Reply with quote

sn4ip3r wrote:
Could anyone test this "defragmenter" and see if it reduces the defragmentation?

I used Con Kolivas's script on a single folder and found that it performed as advertised, though it was quite slow, at least on my machine. You'd want to allow plenty of time to use it on an entire partition. Of course, any defrag utility will be pretty slow, considering that it has to rearrange every bit of data in the target area.

I believe it reduced fragmentation from 5 to 2 percent, or thereabouts. I don't have the results in front of me right now.
_________________
This space available!
Back to top
View user's profile Send private message
sn4ip3r
Guru
Guru


Joined: 14 Dec 2002
Posts: 325
Location: Tallinn, Estonia

PostPosted: Sat Feb 04, 2006 4:30 pm    Post subject: Reply with quote

dhave wrote:
sn4ip3r wrote:
Could anyone test this "defragmenter" and see if it reduces the defragmentation?

I used Con Kolivas's script on a single folder and found that it performed as advertised, though it was quite slow, at least on my machine. You'd want to allow plenty of time to use it on an entire partition. Of course, any defrag utility will be pretty slow, considering that it has to rearrange every bit of data in the target area.

I believe it reduced fragmentation from 5 to 2 percent, or thereabouts. I don't have the results in front of me right now.


On a 200GB partition, which was over 95% full it ran 6h, basically what the script does is copy every file, but it does it in an order which encourages the filesystem to place related files close to eachother.
Back to top
View user's profile Send private message
dhave
Apprentice
Apprentice


Joined: 28 Oct 2005
Posts: 298
Location: Still outside the Matrix ...

PostPosted: Sat Feb 04, 2006 5:43 pm    Post subject: Reply with quote

sn4ip3r wrote:
dhave wrote:

I used Con Kolivas's script on a single folder and found that it performed as advertised, though it was quite slow, at least on my machine. You'd want to allow plenty of time to use it on an entire partition. Of course, any defrag utility will be pretty slow, considering that it has to rearrange every bit of data in the target area.

I believe it reduced fragmentation from 5 to 2 percent, or thereabouts. I don't have the results in front of me right now.


On a 200GB partition, which was over 95% full it ran 6h, basically what the script does is copy every file, but it does it in an order which encourages the filesystem to place related files close to eachother.

That run time sounds pretty typical -- perhaps even a bit faster -- of what I remember of Windows defrag sessions.
_________________
This space available!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum