SATA + Software RAID Performance Doesn't Add Up

Message

Spack · Post by **Spack** » Thu Apr 22, 2004 5:34 am

I just recently moved back to Gentoo after a years abscence. I don't often post here, usually only search and read.

I am currently running gentoo-dev-sources 2.6.3. I have a Promise PDC20376 SATA/RAID controller onboard:

Code: Select all

# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST380013AS       Rev: 1.00
  Type:   Direct-Access                    ANSI SCSI revision: 05
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: ATA      Model: ST380013AS       Rev: 1.00
  Type:   Direct-Access                    ANSI SCSI revision: 05

and wanted to stripe them for performance:

Code: Select all

# cat /proc/mdstat
Personalities : [raid0] [raid1]
md1 : active raid0 sdb3[1] sda3[0]
      155701760 blocks 32k chunks
       
md0 : active raid1 sdb1[1] sda1[0]
      40064 blocks [2/2] [UU]
       
unused devices: <none>

However, I don't seem to be getting the preformance benefit I would expect from the stripped drives:

Code: Select all

# hdparm -tT /dev/sda
 
/dev/sda:
 Timing buffer-cache reads:   892 MB in  2.01 seconds = 444.73 MB/sec
 Timing buffered disk reads:  152 MB in  3.05 seconds =  49.78 MB/sec

Code: Select all

# hdparm -tT /dev/md1
 
/dev/md1:
 Timing buffer-cache reads:   888 MB in  2.01 seconds = 442.74 MB/sec
 Timing buffered disk reads:  168 MB in  3.01 seconds =  55.84 MB/sec

Can anyone shed some light on why my performance might be like this? Am I missing something basic?

Spack · Post by **Spack** » Sat Apr 24, 2004 8:59 am

Bump. So no one's got any ideas?

Crg · Post by **Crg** » Sat Apr 24, 2004 11:29 am

Spack wrote:Bump. So no one's got any ideas?

What does hdparm -t /dev/sdb show?

Spack · Post by **Spack** » Sat Apr 24, 2004 12:42 pm

Code: Select all

# hdparm -tT /dev/sdb
 
/dev/sdb:
 Timing buffer-cache reads:   900 MB in  2.01 seconds = 448.50 MB/sec
 Timing buffered disk reads:  168 MB in  3.03 seconds =  55.44 MB/sec

Spack · Post by **Spack** » Tue Aug 17, 2004 11:02 pm

I've been thinking further about this and I think I might have an answer as to why this is. I ahve two partitions on each physical drive. One partition from each drive is used in a raid 0 configuration for /boot (md0) and the other partition for each drive is in a raid 1 configuration for / (md1).

Is it possible that beacuse I have the md0 in mirror mode this is preventing the md1 (striped) from achieving the performance its capable of?

i.e. if I changed md0 to striped also would the performance change?

I'm keen to try but want to know I'm going down the right path first.

Nate_S · Post by **Nate_S** » Tue Aug 17, 2004 11:42 pm

you may be on the right track there. I always thought you stripe the devices first, then make partitions on top of that, not the other way around. YMMV

sindre · Post by **sindre** » Tue Aug 17, 2004 11:42 pm

I doubt that would work. I've got the same problem, but on P-ATA. I think it's just my IDE-controller sucking though.

/dev/hda and /dev/hdc is set up in raid0 md0.
hdparm -t /dev/hda : about 50MB/s
hdparm -t /dev/hdc : about 50MB/s

hdparm -t /dev/md0 : about 55MB/s

hparm -t /dev/hda & hdparm -t /dev/hdc : about 28 MB/s each.

So maybe my ICH4 intel integrated ide controller can't manage full speed on both IDE buses at the same time? I'd appreciate if anyone with good rad0 scores could do the same test. Anyone know how much overhead in terms of latency and cpu% software raid0 gives? Maybe I'm better off without.

Spack · Post by **Spack** » Wed Aug 18, 2004 2:30 am

Nate_S wrote:you may be on the right track there. I always thought you stripe the devices first, then make partitions on top of that, not the other way around. YMMV

Well yes that is the normal approach with hardware raid however this is using linux software raid. Although I have an onboard Promise 20376 raid controller the software raid is generally recommended as the performance difference is minimal and the software raid is easier to implement.

XioXouS · Post by **XioXouS** » Fri Apr 01, 2005 7:17 pm

After much searching I came across a solution

blockdev --setra 4096 /dev/sda
blockdev --setra 4096 /dev/sdb
blockdev --setra 4096 /dev/md1

Hope that helps you too.

jmahler · Post by **jmahler** » Thu Apr 07, 2005 6:31 pm

XioXouS wrote:After much searching I came across a solution

blockdev --setra 4096 /dev/sda
blockdev --setra 4096 /dev/sdb
blockdev --setra 4096 /dev/md1

Hope that helps you too.

How does one make that a permanent change? it seems to go away after a reboot.

smithjd15 · Post by **smithjd15** » Thu Apr 07, 2005 8:52 pm

How does one make that a permanent change? it seems to go away after a reboot.

Better question...what does it do?

XioXouS · Post by **XioXouS** » Sun Apr 10, 2005 8:24 pm

/etc/conf.d/local is a good place for personal customizations like that

man blockdev

this particular command sets the readahead amount higher in the hopes that it will get a little more usable data at a time and not need to access the disk again for a bit, of course it doesn't work very well if the things you wanted weren't in order on the disk, and I don't believe hdparm checks that very well

i've heard, but not tested yet, that bonnie++ is a better way to measure actual performance

did it work for you too, in terms of hdparm -Tt results?