Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Performance tuning 3ware 9000 Series Cards
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
Ast0r
Guru
Guru


Joined: 11 Apr 2006
Posts: 404
Location: Dallas, Tx - USA

PostPosted: Wed Aug 16, 2006 4:59 pm    Post subject: Performance tuning 3ware 9000 Series Cards Reply with quote

If you have a 3ware 9000 series card, you might be missing out on some of the performance that your card/drives are capable of. For the last 4 months I have been very frustrated becasue the read-speed on my RAID5 arrays was abysmal. On my 1.2TB array (5x300GB in RAID5 on a 3ware 9550SX-8) I was getting only about 65MB/s on sequential reads (via "hdparm -tT /dev/sda"). On my 900GB array (4x300GB in RAID5 on a 3ware 9000S) I was getting around 90MB/s. Obviously something was wrong, but I could never figure out what it was!

Well, now I know: I needed to adjust the kernel's read-ahead cache settings for the device. According to the 3ware white-paper that I downloaded, you can do this as follows:
Code:
blockdev --setra 16384 /dev/sda


Once you do this, add the same line to /etc/conf.d/local.start so that the change will persist across reboots.

Remeber how I said I was getting 65MB/s on the 5 disk array? Check it out now:
Code:
yang ~ # hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   3532 MB in  2.00 seconds = 1765.89 MB/sec
 Timing buffered disk reads:  656 MB in  3.06 seconds = 214.65 MB/sec


Sweet!


Last edited by Ast0r on Tue Oct 31, 2006 10:20 pm; edited 1 time in total
Back to top
View user's profile Send private message
nianderson
Guru
Guru


Joined: 06 May 2003
Posts: 369
Location: Lawrence, KS

PostPosted: Tue Oct 31, 2006 8:44 pm    Post subject: Reply with quote

But 3ware claims faster speeds.

The 3ware 9550SX PCI-X to SATA II RAID controller delivers over 800MB/sec RAID 5 reads and exceeds 380MB/sec RAID 5 writes, making it 200% faster than the industry-leading 3ware 9500S RAID controller.

Ive got a customer with a 9550sx and he is claiming this output.

Code:

 hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   3684 MB in  2.00 seconds = 1840.44 MB/sec
 Timing buffered disk reads:  296 MB in  3.01 seconds =  98.48 MB/sec
Back to top
View user's profile Send private message
Ast0r
Guru
Guru


Joined: 11 Apr 2006
Posts: 404
Location: Dallas, Tx - USA

PostPosted: Tue Oct 31, 2006 10:19 pm    Post subject: Reply with quote

nianderson wrote:
But 3ware claims faster speeds.

The 3ware 9550SX PCI-X to SATA II RAID controller delivers over 800MB/sec RAID 5 reads and exceeds 380MB/sec RAID 5 writes, making it 200% faster than the industry-leading 3ware 9500S RAID controller.

Ive got a customer with a 9550sx and he is claiming this output.

Code:

 hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   3684 MB in  2.00 seconds = 1840.44 MB/sec
 Timing buffered disk reads:  296 MB in  3.01 seconds =  98.48 MB/sec

How many drives? Keep in mind that the 800MB/s top read speed is the total capability of the card. Right now my setup is limited by the speed of the disks (7200rpm) and the fact that I am using only 5 (whereas it supports 8). If you used (8) 15k RPM disks, your transfer rate should be a LOT better. Obviously, your mileage will vary according to the hardware that you connect to it.

So did your client try this? Did he get any improvement?
Back to top
View user's profile Send private message
nianderson
Guru
Guru


Joined: 06 May 2003
Posts: 369
Location: Lawrence, KS

PostPosted: Tue Oct 31, 2006 10:26 pm    Post subject: Reply with quote

the blockdev --setra we have been using for a long time. It does indeed increase performance.

Im just trying to figure out why he has one golden machine that performs 2 times as good as others we have sent him.
Back to top
View user's profile Send private message
Ast0r
Guru
Guru


Joined: 11 Apr 2006
Posts: 404
Location: Dallas, Tx - USA

PostPosted: Tue Oct 31, 2006 10:36 pm    Post subject: Reply with quote

nianderson wrote:
the blockdev --setra we have been using for a long time. It does indeed increase performance.

Im just trying to figure out why he has one golden machine that performs 2 times as good as others we have sent him.

Maybe the hard drives are better in that particular machine?
Back to top
View user's profile Send private message
truE_
n00b
n00b


Joined: 22 Nov 2006
Posts: 3

PostPosted: Wed Nov 22, 2006 8:44 am    Post subject: Reply with quote

I'm getting unbearable performence from my 3ware 9550SX-4LP. I'm using 3ware's latest firmware (as of today), kernel 2.6.18.3's built-in scsi driver, and Seagate ST3750640AS 750GB drives in a raid5 configuration. (1.6TB) I'm getting write speeds of 3MB/s, and read speeds of around 25MB/s:

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 2064 MB in 2.00 seconds = 1031.50 MB/sec
Timing buffered disk reads: 74 MB in 3.03 seconds = 24.39 MB/sec

I've replaced the backplane, tried other drives (same model), and multiple different kernel versions. Same thing...

I did the blockdev command as a test and speeds did not change.

Anyone have any ideas? I'm wondering if its these drives...
Back to top
View user's profile Send private message
Ast0r
Guru
Guru


Joined: 11 Apr 2006
Posts: 404
Location: Dallas, Tx - USA

PostPosted: Wed Nov 22, 2006 9:08 am    Post subject: Reply with quote

Perhaps you have a configuration problem on your card? You have you tried experimenting with the settings in the 3ware card's BIOS?
Back to top
View user's profile Send private message
nianderson
Guru
Guru


Joined: 06 May 2003
Posts: 369
Location: Lawrence, KS

PostPosted: Wed Nov 22, 2006 2:04 pm    Post subject: Reply with quote

Do you have a BBU?
If you dont, I am not positive that turning write-caching on will actually do anything. Ive seen that before but anyway try the following
Download both tw_cli and 3dm2. Check for errors in 3dm.
In tw_cli set storsave to perform, set queuing to off.

What motherboard is this in? Are you using a riser?
Back to top
View user's profile Send private message
truE_
n00b
n00b


Joined: 22 Nov 2006
Posts: 3

PostPosted: Wed Nov 22, 2006 6:26 pm    Post subject: Reply with quote

Thanks for all of your help and suggestions -- I am working dilegently to resolve this.

I was getting very poor performance with the default BIOS options (I simply did a Firmware upgrade when first installed).

I am still getting very poor performance after doing what you guys suggested:

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 2088 MB in 2.00 seconds = 1042.61 MB/sec
Timing buffered disk reads: 90 MB in 3.03 seconds = 29.71 MB/sec

This is with storsave set to "Performance", Queuing "Off", and write-caching "off". Since I don't have a BBU (I ordered one, but got shipped the 9500S one), write caching should really make no performance differences.

I'm using an Intel SR1450 Chassis, with an Intel SE7520JR2DDR2 motherboard and an Intel PCIX riser (ADWPCIXR).

Like I said, I've tried different drive sets (of the same model drive), another backplane, and all new SATA cables. Maybe the riser is hindering performance?

Any other ideas from anyone? I am using the exact same configuration with SCSI drives and LSILogic raid controllers, and the machines are smoking fast. I am also getting tons of CPU when doing writes. I'll include a few more examples below:

My RAID1 LSI SCSI machines:

time dd if=/dev/zero of=testfile bs=1024k count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.16581 seconds, 496 MB/s

real 0m2.168s
user 0m0.000s
sys 0m2.170s

My RAID5 3WARE SATA PCI-X

Still running after 10+ MINUTES (took 2 seconds with the other machine!!) w/ an 8+ CPU Load.... Will update this thread if/when it finishes.

Of course I expect a drastic difference between SCSI and Raid 1 vs. 5 -- but this is tragic, something hardware-wise must not be in good shape...

Thank you very much.
Back to top
View user's profile Send private message
nianderson
Guru
Guru


Joined: 06 May 2003
Posts: 369
Location: Lawrence, KS

PostPosted: Wed Nov 22, 2006 6:54 pm    Post subject: Reply with quote

I would try bypassing the riser, we have seen numerous issues with risers.
3ware does reccomend active risers, is that riser passive or active?

In fact I have a 3ware 9550sx series with 4 drives in our lab right now.

Its in the middle of a test but Ill get the hdparm output for ya as well as the drive models
Back to top
View user's profile Send private message
truE_
n00b
n00b


Joined: 22 Nov 2006
Posts: 3

PostPosted: Wed Nov 22, 2006 9:25 pm    Post subject: Reply with quote

nianderson wrote:
I would try bypassing the riser, we have seen numerous issues with risers.
3ware does reccomend active risers, is that riser passive or active?

In fact I have a 3ware 9550sx series with 4 drives in our lab right now.

Its in the middle of a test but Ill get the hdparm output for ya as well as the drive models


Thanks again, I appreciate it.

I will try and remove the riser and see if there is any difference. I could not find if it was an Active or Passive riser -- for reference, here is a link to it through newegg.com (for picture reasons)

http://www.newegg.com/Product/Product.asp?Item=N82E16816117029

Lastly, I would greatly appreciate an hdparm w/ raid 5 using that controller. (also, is it PCI-X?)
Back to top
View user's profile Send private message
nianderson
Guru
Guru


Joined: 06 May 2003
Posts: 369
Location: Lawrence, KS

PostPosted: Wed Nov 22, 2006 10:24 pm    Post subject: Reply with quote

That looks like a passive riser to me.

The machine I have in the lab is still running heavy disk tests.
so this is under heavy disk load.

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 10200 MB in 2.00 seconds = 5100.78 MB/sec
Timing buffered disk reads: 54 MB in 3.01 seconds = 17.92 MB/sec

Unit UnitType Status %Cmpl Stripe Size(GB) Cache AVerify IgnECC
------------------------------------------------------------------------------
u0 RAID-5 VERIFYING 18 64K 419.065 OFF OFF OFF

Port Status Unit Size Blocks Serial
---------------------------------------------------------------
p0 OK u0 139.73 GB 293046768 WD-WMAP41189376
p1 OK u0 139.73 GB 293046768 WD-WMAP41142747
p2 OK u0 139.73 GB 293046768 WD-WMAP41146945
p3 OK u0 139.73 GB 293046768 WD-WMAP41184828

as you notice its also verifying so that bench is pretty much useless sorry :)
Ill try to remeber to get the bench and post it before we ship it out.
Back to top
View user's profile Send private message
-Craig-
Guru
Guru


Joined: 03 Jun 2004
Posts: 333

PostPosted: Sun Dec 24, 2006 12:10 am    Post subject: Reply with quote

You *REALLY* need to enable write caching, it makes a *BIG* difference!!!!
Back to top
View user's profile Send private message
WhiteSpade
Tux's lil' helper
Tux's lil' helper


Joined: 16 Apr 2005
Posts: 89

PostPosted: Thu Jan 25, 2007 7:19 am    Post subject: Reply with quote

I've been looking around 3ware's site and I found this:
Quote:
blockdev --setra 16384 /dev/sda
(Note: 16384 is just an example value. You will have to do testing to determine the optimal value for your system). The OS will read-ahead X pages, and throughput will be higher.

So... I am looking forward to experimenting with a few different values. I know some people have achieved enormous gains using the suggested value while others have only seen modest gains. I wonder if different values will help. I'll let you all know if I find anything.

---Alex
Back to top
View user's profile Send private message
Ast0r
Guru
Guru


Joined: 11 Apr 2006
Posts: 404
Location: Dallas, Tx - USA

PostPosted: Thu Jan 25, 2007 8:10 am    Post subject: Reply with quote

WhiteSpade wrote:
I've been looking around 3ware's site and I found this:
Quote:
blockdev --setra 16384 /dev/sda
(Note: 16384 is just an example value. You will have to do testing to determine the optimal value for your system). The OS will read-ahead X pages, and throughput will be higher.

So... I am looking forward to experimenting with a few different values. I know some people have achieved enormous gains using the suggested value while others have only seen modest gains. I wonder if different values will help. I'll let you all know if I find anything.

---Alex

A friend of mine wrote a shell script to try different values and figure out which is best. What we determined was that 16384 is hard to beat in terms of overall performance, at least on the 9xxx series cards that we have.

If you are interested, here is the shell script that he wrote:
Code:
#!/bin/bash

START=0
INCREMENT=16
ALLOWDRIFT="50.0"

speed=0
try=$(( $START - $INCREMENT ))
keepgoing=true
topspeed=0
topra=0
while [[ "${keepgoing}" == "true" ]]; do
   try=$(( $try + $INCREMENT ))
   echo -n "Trying $try readahead..."
   blockdev --setra $try /dev/sda
   speed=$(hdparm -t /dev/sda | tail -n 1 | awk '{print $11}')
   echo -n " ${speed} MB/sec"

# Hmm...
   if [ "$(echo "scale=3; ${speed} > ${topspeed}" | bc )" -eq '1' ]; then
      echo -e "\t** TOP SPEED"
      topspeed="${speed}"
      topra="${try}"
      keepgoing=true
   else
      percdiff=$(echo -e "scale=3\n100 - ( ( ${speed} / ${topspeed} ) * 100 )\nquit" | bc)
      if [ "$(echo "scale=3; ${percdiff} < ${ALLOWDRIFT}" | bc)" -eq '1' ]; then
         echo -e "\t${percdiff}% slower"
         keepgoing=true
      else
         echo -e "\t${percdiff}% slower, bailing..."
         keepgoing=false
      fi
   fi

done

echo -e "Looks like the best speed of ${topspeed} MB/sec is achieved with a readahead of ${topra}.\n"


I would be interested if you find that another value works better for you. If so, please post which card you are using, how many drives, array configuration, etc.

Also, keep in mind that this script only increments by 16 blocks each recursion and the hdparm takes about 12 seconds per run, so you need about 12 * 2000 seconds just to hit 32768 (double the recommended value from 3ware). You will either want to change the granularity or run this over night (I recommend the latter if you are serious about finding the best value).
Back to top
View user's profile Send private message
-Craig-
Guru
Guru


Joined: 03 Jun 2004
Posts: 333

PostPosted: Sun Feb 11, 2007 8:19 pm    Post subject: Reply with quote

For me, "blockdev --setra 16384 /dev/sda" doesn't make any difference.
Back to top
View user's profile Send private message
dtlgc
n00b
n00b


Joined: 27 Jul 2004
Posts: 60
Location: Texas

PostPosted: Wed Mar 07, 2007 2:48 am    Post subject: 3ware Before & After Reply with quote

My before and after on 3ware card - plain mirrored drives

ic conf.d # hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 4308 MB in 2.00 seconds = 2153.87 MB/sec
Timing buffered disk reads: 164 MB in 3.02 seconds = 54.37 MB/sec
ic conf.d # blockdev --setra 16384 /dev/sda
ic conf.d # hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 4328 MB in 2.00 seconds = 2163.86 MB/sec
Timing buffered disk reads: 240 MB in 3.01 seconds = 79.68 MB/sec
Back to top
View user's profile Send private message
col
l33t
l33t


Joined: 08 May 2002
Posts: 819
Location: Melbourne - Australia

PostPosted: Tue May 08, 2007 2:06 am    Post subject: Reply with quote

-Craig- wrote:
You *REALLY* need to enable write caching, it makes a *BIG* difference!!!!


I remember reading somewhere that this can be very dangerous if your using a journaling file system ?
Back to top
View user's profile Send private message
Bo_Oris
n00b
n00b


Joined: 07 Jul 2005
Posts: 21

PostPosted: Sat May 19, 2007 3:38 pm    Post subject: verschoben Reply with quote

Verschoben
_________________
Asus P5WDG2 Pro
Intel Core 2 Duo E6600 @ 2,4 Ghz
2 x OCZ 1024MB 4-4-4-12
3 x 300 GB Sata 2 @ Raid 5
1 x 3com 9550SX-4LP Raid Controler

Kernel: 2.6.21.14
Desktop: KDE @ Beryl auf 2 x 22" Widescreen TFT
Back to top
View user's profile Send private message
MentholMoose
n00b
n00b


Joined: 29 Jan 2007
Posts: 1
Location: USA

PostPosted: Sun May 20, 2007 8:14 pm    Post subject: Reply with quote

Thanks for the tip. 9550SX-12 with 7x 160GB Seagate drives.

root@knightboat ~ # blockdev --getra /dev/sda
256
root@knightboat ~ # hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 1864 MB in 2.00 seconds = 932.62 MB/sec
Timing buffered disk reads: 368 MB in 3.00 seconds = 122.61 MB/sec
root@knightboat ~ # blockdev --setra 16384 /dev/sda
root@knightboat ~ # hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 1882 MB in 2.00 seconds = 941.10 MB/sec
Timing buffered disk reads: 672 MB in 3.01 seconds = 223.39 MB/sec
Back to top
View user's profile Send private message
NightMonkey
Guru
Guru


Joined: 21 Mar 2003
Posts: 317
Location: San Francisco, CA

PostPosted: Sat Dec 22, 2007 1:44 am    Post subject: Reply with quote

Some very interesting legwork done here:
* http://www.makarevitch.org/rant/3ware/

And some interesting discussion and comparative benchmarks here:
* http://forums.storagereview.net/index.php?showtopic=25923&pid=245225&mode=threaded&start=#entry245225
_________________
I like Linux. :D
Back to top
View user's profile Send private message
fixinko
n00b
n00b


Joined: 23 Jun 2007
Posts: 16
Location: Bratislava, Slovakia

PostPosted: Wed Mar 19, 2008 1:26 pm    Post subject: Reply with quote

I've using 9550SXU-4 with 4xWDC5001ABYS (RAID-5). Read performance is quite good, but on database load or moving big files (write operations), load goes to 5-12 (big i/o wait), and the whole system is lazy. I've read this http://www.3ware.com/KB/article.aspx?id=11050 article, but those settings, doesn't solve my problem. What did I wrong? My actual settings:
hdparm:
Code:

phoenix ~ # hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   6388 MB in  2.00 seconds = 3196.67 MB/sec
 Timing buffered disk reads:  336 MB in  3.02 seconds = 111.25 MB/sec

tw_cli:
Code:

//phoenix> /c0 show

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-5    OK             -       -       64K     1396.95   ON     OFF

Port   Status           Unit   Size        Blocks        Serial
---------------------------------------------------------------
p0     OK               u0     465.76 GB   976773168     WD-WCAS83789493
p1     OK               u0     465.76 GB   976773168     WD-WCAS83782829
p2     OK               u0     465.76 GB   976773168     WD-WCAS83799487
p3     OK               u0     465.76 GB   976773168     WD-WCAS85160709

//phoenix> /c0 show all
/c0 Driver Version = 2.26.02.010
/c0 Model = 9550SXU-4LP
/c0 Available Memory = 112MB
/c0 Firmware Version = FE9X 3.04.00.005
/c0 Bios Version = BE9X 3.04.00.002
/c0 Boot Loader Version = BL9X 3.02.00.001
/c0 Serial Number = L320910A7441596
/c0 PCB Version = Rev 032
/c0 PCHIP Version = 1.60
/c0 ACHIP Version = 1.90
/c0 Number of Ports = 4
/c0 Number of Drives = 4
/c0 Number of Units = 1
/c0 Total Optimal Units = 1
/c0 Not Optimal Units = 0
/c0 JBOD Export Policy = off
/c0 Disk Spinup Policy = 1
/c0 Spinup Stagger Time Policy (sec) = 2
/c0 Auto-Carving Policy = off
/c0 Auto-Carving Size = 2048 GB
/c0 Auto-Rebuild Policy = on
/c0 Controller Bus Type = PCIX
/c0 Controller Bus Width = 64 bits
/c0 Controller Bus Speed = 133 Mhz

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-5    OK             -       -       64K     1396.95   ON     OFF

Port   Status           Unit   Size        Blocks        Serial
---------------------------------------------------------------
p0     OK               u0     465.76 GB   976773168     WD-WCAS83789493
p1     OK               u0     465.76 GB   976773168     WD-WCAS83782829
p2     OK               u0     465.76 GB   976773168     WD-WCAS83799487
p3     OK               u0     465.76 GB   976773168     WD-WCAS85160709


blockdev and /sys:
Code:

phoenix ~ # cat /sys/block/sda/queue/max_sectors_kb
128

phoenix ~ # cat /sys/block/sda/queue/nr_requests
8192

phoenix ~ # cat /proc/sys/vm/page-cluster
3

phoenix ~ # blockdev --report /dev/sda
RO    RA   SSZ   BSZ   StartSec     Size    Device
rw 16384   512  4096          0 2929625088  /dev/sda

phoenix ~ # cat /proc/sys/vm/dirty_background_ratio
20

phoenix ~ # cat /proc/sys/vm/dirty_ratio
60

phoenix ~ # cat /sys/block/sda/queue/scheduler
noop anticipatory [deadline]


/proc/interrupts:
Code:

phoenix ~ # cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
  0:   58053879          0          0          0   IO-APIC-edge      timer
  1:          2          0          0          0   IO-APIC-edge      i8042
  4:        165          0          0          0   IO-APIC-edge      serial
  8:          0          0          0          0   IO-APIC-edge      rtc
  9:          0          0          0          0   IO-APIC-fasteoi   acpi
 12:          3          0          0          0   IO-APIC-edge      i8042
 14:         78          0          0          0   IO-APIC-edge      ide0
 19:          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
 23:          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb2
 24:    6256880          0          0          0   IO-APIC-fasteoi   3w-9xxx
380:   65061197          0          0          0   PCI-MSI-edge      eth0
NMI:          0          0          0          0
LOC:   57333954   57333952   57534123   57533999
ERR:          0


uname:
Code:

phoenix ~ # uname -a
Linux phoenix 2.6.23-hardened-r8 #1 SMP Mon Mar 17 08:10:52 CET 2008 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux
Back to top
View user's profile Send private message
NightMonkey
Guru
Guru


Joined: 21 Mar 2003
Posts: 317
Location: San Francisco, CA

PostPosted: Wed Mar 19, 2008 6:13 pm    Post subject: IOZone Reply with quote

Try using IOZone as your benchmarking tool. It is in Portage. All of the tools you've used are read-only tools. You have no data shown about your write performance.

FYI: I just kicked some 3ware cards out of the servers I was deploying, and replaced them with PCI-Express Areca cards. Incredible improvement in performance, both write and read, especially random write performance, which was our problem area (RAID 5). It doesn't work miracles, but it doesn't put much in the way between Linux and the disks. If I have a choice, I'll not choose 3ware again until I hear they've cleaned up their act.

fixinko wrote:
I've using 9550SXU-4 with 4xWDC5001ABYS (RAID-5). Read performance is quite good, but on database load or moving big files (write operations), load goes to 5-12 (big i/o wait), and the whole system is lazy. I've read this http://www.3ware.com/KB/article.aspx?id=11050 article, but those settings, doesn't solve my problem. What did I wrong? My actual settings:
hdparm:
Code:

phoenix ~ # hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   6388 MB in  2.00 seconds = 3196.67 MB/sec
 Timing buffered disk reads:  336 MB in  3.02 seconds = 111.25 MB/sec

tw_cli:
Code:

//phoenix> /c0 show

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-5    OK             -       -       64K     1396.95   ON     OFF

Port   Status           Unit   Size        Blocks        Serial
---------------------------------------------------------------
p0     OK               u0     465.76 GB   976773168     WD-WCAS83789493
p1     OK               u0     465.76 GB   976773168     WD-WCAS83782829
p2     OK               u0     465.76 GB   976773168     WD-WCAS83799487
p3     OK               u0     465.76 GB   976773168     WD-WCAS85160709

//phoenix> /c0 show all
/c0 Driver Version = 2.26.02.010
/c0 Model = 9550SXU-4LP
/c0 Available Memory = 112MB
/c0 Firmware Version = FE9X 3.04.00.005
/c0 Bios Version = BE9X 3.04.00.002
/c0 Boot Loader Version = BL9X 3.02.00.001
/c0 Serial Number = L320910A7441596
/c0 PCB Version = Rev 032
/c0 PCHIP Version = 1.60
/c0 ACHIP Version = 1.90
/c0 Number of Ports = 4
/c0 Number of Drives = 4
/c0 Number of Units = 1
/c0 Total Optimal Units = 1
/c0 Not Optimal Units = 0
/c0 JBOD Export Policy = off
/c0 Disk Spinup Policy = 1
/c0 Spinup Stagger Time Policy (sec) = 2
/c0 Auto-Carving Policy = off
/c0 Auto-Carving Size = 2048 GB
/c0 Auto-Rebuild Policy = on
/c0 Controller Bus Type = PCIX
/c0 Controller Bus Width = 64 bits
/c0 Controller Bus Speed = 133 Mhz

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-5    OK             -       -       64K     1396.95   ON     OFF

Port   Status           Unit   Size        Blocks        Serial
---------------------------------------------------------------
p0     OK               u0     465.76 GB   976773168     WD-WCAS83789493
p1     OK               u0     465.76 GB   976773168     WD-WCAS83782829
p2     OK               u0     465.76 GB   976773168     WD-WCAS83799487
p3     OK               u0     465.76 GB   976773168     WD-WCAS85160709


blockdev and /sys:
Code:

phoenix ~ # cat /sys/block/sda/queue/max_sectors_kb
128

phoenix ~ # cat /sys/block/sda/queue/nr_requests
8192

phoenix ~ # cat /proc/sys/vm/page-cluster
3

phoenix ~ # blockdev --report /dev/sda
RO    RA   SSZ   BSZ   StartSec     Size    Device
rw 16384   512  4096          0 2929625088  /dev/sda

phoenix ~ # cat /proc/sys/vm/dirty_background_ratio
20

phoenix ~ # cat /proc/sys/vm/dirty_ratio
60

phoenix ~ # cat /sys/block/sda/queue/scheduler
noop anticipatory [deadline]


/proc/interrupts:
Code:

phoenix ~ # cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
  0:   58053879          0          0          0   IO-APIC-edge      timer
  1:          2          0          0          0   IO-APIC-edge      i8042
  4:        165          0          0          0   IO-APIC-edge      serial
  8:          0          0          0          0   IO-APIC-edge      rtc
  9:          0          0          0          0   IO-APIC-fasteoi   acpi
 12:          3          0          0          0   IO-APIC-edge      i8042
 14:         78          0          0          0   IO-APIC-edge      ide0
 19:          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
 23:          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb2
 24:    6256880          0          0          0   IO-APIC-fasteoi   3w-9xxx
380:   65061197          0          0          0   PCI-MSI-edge      eth0
NMI:          0          0          0          0
LOC:   57333954   57333952   57534123   57533999
ERR:          0


uname:
Code:

phoenix ~ # uname -a
Linux phoenix 2.6.23-hardened-r8 #1 SMP Mon Mar 17 08:10:52 CET 2008 x86_64 Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz GenuineIntel GNU/Linux

_________________
I like Linux. :D
Back to top
View user's profile Send private message
fixinko
n00b
n00b


Joined: 23 Jun 2007
Posts: 16
Location: Bratislava, Slovakia

PostPosted: Wed Mar 19, 2008 6:49 pm    Post subject: Re: IOZone Reply with quote

NightMonkey wrote:
Try using IOZone as your benchmarking tool. It is in Portage. All of the tools you've used are read-only tools. You have no data shown about your write performance.

FYI: I just kicked some 3ware cards out of the servers I was deploying, and replaced them with PCI-Express Areca cards. Incredible improvement in performance, both write and read, especially random write performance, which was our problem area (RAID 5). It doesn't work miracles, but it doesn't put much in the way between Linux and the disks. If I have a choice, I'll not choose 3ware again until I hear they've cleaned up their act.

I haven't problem with write performance, but with big i/o waits on write operations, so therefore I didn't show data about my write performance...
Back to top
View user's profile Send private message
NightMonkey
Guru
Guru


Joined: 21 Mar 2003
Posts: 317
Location: San Francisco, CA

PostPosted: Wed Mar 19, 2008 7:24 pm    Post subject: Re: IOZone Reply with quote

If you haven't already, see my links posted above ( http://forums.storagereview.net/index.php?showtopic=25923&pid=249105&mode=threaded&start=#entry249105 ) . Seems to deal specifically with the symptoms you are describing.

fixinko wrote:
NightMonkey wrote:
Try using IOZone as your benchmarking tool. It is in Portage. All of the tools you've used are read-only tools. You have no data shown about your write performance.

FYI: I just kicked some 3ware cards out of the servers I was deploying, and replaced them with PCI-Express Areca cards. Incredible improvement in performance, both write and read, especially random write performance, which was our problem area (RAID 5). It doesn't work miracles, but it doesn't put much in the way between Linux and the disks. If I have a choice, I'll not choose 3ware again until I hear they've cleaned up their act.

I haven't problem with write performance, but with big i/o waits on write operations, so therefore I didn't show data about my write performance...

_________________
I like Linux. :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum