| View previous topic :: View next topic |
| Author |
Message |
urgh
n00b
Joined: 16 Apr 2012
Posts: 5
|
 Posted: Mon Apr 16, 2012 11:15 am Post subject: Slow raid |
 |
|
Hello,
I've installed Gentoo on my older machine that used to run Win server 2008. Machine has one hard drive for system and two drives for storing data that is not so important (I run raid0 on them). Using software raid on Windows I got around 80~100 mb/s read from soft raid.
Now, after installing Gentoo and setting up drives with mdadm I get 30 mb/s max?! What did I do wrong?
This is what I used to set up raid:
| Code: | mdadm --create /dev/md0 --chunk=128 --level=0 --raid-devices=2 /dev/sd[bc]1
mkfs.ext3 -b 4096 -E stride=32,stripe-width=64 /dev/md0
|
| Code: | # cat /proc/mdstat
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid0 sdb1[0] sdc1[1]
234435328 blocks super 1.2 128k chunks
unused devices: <none>
|
Ask If you need any other info. |
|
| Back to top |
|
 |
Mad Merlin
Veteran
Joined: 09 May 2005
Posts: 1134
|
| Posted: Tue Apr 17, 2012 12:24 am Post subject: |
|
|
Three questions. First, how are you benchmarking? Second, does the result change if you use the default settings for the array and filesystem (ie, drop the chunk size, block size and -E options)? Lastly, if you benchmark the drives individually, what kind of numbers do you get?
_________________
Game! - Where the stick is mightier than the sword! |
|
| Back to top |
|
|
urgh
n00b
Joined: 16 Apr 2012
Posts: 5
|
| Posted: Tue Apr 17, 2012 4:06 pm Post subject: |
|
|
Bonnie gives this results:
| Code: | # bonnie -s 2000
File './Bonnie.22404', size: 2097152000
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
2000 45114 99.2 90634 22.6 89068 11.9 47431 100.0 2423886 100.4 123399.7 111.1
|
but copying large 8 gig file from raid to system harddrive in Midnight Commander gives around 30 mb/s. Copying from Windows through network is around the same speed.
First I've tried default settings (no extra params for chunk, stripe, stride) and results were the same.
hdparam prints out results that I had on Windows, but I can't get these numbers now:
| Code: | # hdparm -Tt /dev/md0
/dev/md0:
Timing cached reads: 2338 MB in 2.00 seconds = 1169.15 MB/sec
Timing buffered disk reads: 346 MB in 3.01 seconds = 114.90 MB/sec
# hdparm -Tt /dev/sdb1
/dev/sdb1:
Timing cached reads: 2316 MB in 2.00 seconds = 1157.77 MB/sec
Timing buffered disk reads: 174 MB in 3.03 seconds = 57.48 MB/sec
# hdparm -Tt /dev/sdc1
/dev/sdc1:
Timing cached reads: 2336 MB in 2.00 seconds = 1168.21 MB/sec
Timing buffered disk reads: 190 MB in 3.01 seconds = 63.19 MB/sec
|
|
|
| Back to top |
|
|
Mad Merlin
Veteran
Joined: 09 May 2005
Posts: 1134
|
| Posted: Wed Apr 18, 2012 4:13 am Post subject: |
|
|
Your bonnie numbers look about right (~89M/s sequential write).
30M/s sounds like a PCI bus limitation, the embedded gigabit ethernet cards on cheaper motherboards are often connected via PCI and not PCIe, giving them a maximum of ~30M/s. The other hard drive I'm not sure about, but I doubt you'll be able to max out the RAID when copying from a single drive. Try something like so:
| Code: |
dd if=/dev/zero of=bigfile bs=1M count=$[1024*16]
|
That writes a 16G file, make sure that's at least twice the size of your ram, otherwise you're really just testing your memory speed.
_________________
Game! - Where the stick is mightier than the sword! |
|
| Back to top |
|
|
urgh
n00b
Joined: 16 Apr 2012
Posts: 5
|
| Posted: Wed Apr 18, 2012 10:18 am Post subject: |
|
|
You are right, when testing with dd, I got same speed like on previous system:
| Code: | # dd if=/dev/zero of=bigfile bs=1M count=$[1024*16]
16384+0 records in
16384+0 records out
17179869184 bytes (17 GB) copied, 200.318 s, 85.8 MB/s
# dd if=bigfile of=/dev/zero bs=1M count=$[1024*16]
16384+0 records in
16384+0 records out
17179869184 bytes (17 GB) copied, 150.246 s, 114 MB/s |
How can I check speed of my network card? This used to work fine, so maybe I'm missing some setting.
| Code: | # ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: Symmetric Receive-only
Link partner advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
drv probe ifdown ifup
Link detected: yes
# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:1a:4d:93:5a:8f
inet addr:192.168.1.102 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1245 errors:0 dropped:0 overruns:0 frame:0
TX packets:1387 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:274832 (268.3 KiB) TX bytes:234564 (229.0 KiB)
Interrupt:44 Base address:0xc000 |
|
|
| Back to top |
|
|
urgh
n00b
Joined: 16 Apr 2012
Posts: 5
|
| Posted: Wed Apr 18, 2012 10:55 am Post subject: |
|
|
iperf gives this info:
TCP:
| Code: | # iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.1.102 port 5001 connected with 192.168.1.101 port 49544
------------------------------------------------------------
Client connecting to 192.168.1.101, TCP port 5001
TCP window size: 90.9 KByte (default)
------------------------------------------------------------
[ 6] local 192.168.1.102 port 53188 connected with 192.168.1.101 port 5001
Waiting for server threads to complete. Interrupt again to force quit.
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 197 MBytes 165 Mbits/sec
[ 6] 0.0-10.0 sec 946 MBytes 793 Mbits/sec |
| Code: | c:\>iperf -c echo2 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to echo2, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[396] local 192.168.1.101 port 49544 connected with 192.168.1.102 port 5001
[400] local 192.168.1.101 port 5001 connected with 192.168.1.102 port 53188
[ ID] Interval Transfer Bandwidth
[400] 0.0-10.0 sec 946 MBytes 793 Mbits/sec
[396] 0.0-10.0 sec 197 MBytes 165 Mbits/sec |
UDP:
| Code: | # iperf -s -u -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 160 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.1.101, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 160 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.102 port 5001 connected with 192.168.1.101 port 50826
[ 5] local 192.168.1.102 port 49125 connected with 192.168.1.101 port 5001
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 11.4 MBytes 95.7 Mbits/sec 0.104 ms 0/ 8134 (0%)
[ 5] 0.0- 1.0 sec 95.9 MBytes 804 Mbits/sec
[ 3] 1.0- 2.0 sec 6.74 MBytes 56.5 Mbits/sec 0.442 ms 0/ 4806 (0%)
[ 5] 1.0- 2.0 sec 96.2 MBytes 807 Mbits/sec
[ 3] 2.0- 3.0 sec 3.92 MBytes 32.9 Mbits/sec 0.467 ms 0/ 2794 (0%)
[ 5] 2.0- 3.0 sec 96.3 MBytes 808 Mbits/sec
[ 3] 3.0- 4.0 sec 3.91 MBytes 32.8 Mbits/sec 0.469 ms 0/ 2792 (0%)
[ 5] 3.0- 4.0 sec 96.1 MBytes 806 Mbits/sec
[ 3] 4.0- 5.0 sec 3.91 MBytes 32.8 Mbits/sec 0.464 ms 0/ 2786 (0%)
[ 5] 4.0- 5.0 sec 95.9 MBytes 804 Mbits/sec
[ 3] 5.0- 6.0 sec 3.92 MBytes 32.9 Mbits/sec 0.459 ms 0/ 2794 (0%)
[ 5] 5.0- 6.0 sec 96.3 MBytes 808 Mbits/sec
[ 3] 6.0- 7.0 sec 3.90 MBytes 32.8 Mbits/sec 0.456 ms 0/ 2785 (0%)
[ 5] 6.0- 7.0 sec 96.4 MBytes 808 Mbits/sec
[ 3] 7.0- 8.0 sec 3.91 MBytes 32.8 Mbits/sec 0.457 ms 0/ 2789 (0%)
[ 5] 7.0- 8.0 sec 95.8 MBytes 804 Mbits/sec
[ 3] 8.0- 9.0 sec 4.01 MBytes 33.6 Mbits/sec 0.438 ms 0/ 2861 (0%)
[ 5] 8.0- 9.0 sec 96.1 MBytes 806 Mbits/sec
[ 3] 9.0-10.0 sec 3.96 MBytes 33.2 Mbits/sec 0.459 ms 0/ 2823 (0%)
[ 3] 0.0-10.0 sec 49.6 MBytes 41.6 Mbits/sec 0.472 ms 0/35365 (0%)
[ 5] 9.0-10.0 sec 95.9 MBytes 804 Mbits/sec
[ 5] 0.0-10.0 sec 961 MBytes 805 Mbits/sec
[ 5] Sent 685438 datagrams
[ 5] Server Report:
[ 5] 0.0-10.0 sec 311 MBytes 261 Mbits/sec 0.084 ms 463489/685436 (68%)
[ 5] 0.0-10.0 sec 1 datagrams received out-of-order |
| Code: | c:\>iperf -c echo2 -d -u -b 1g
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 8.00 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to echo2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 8.00 KByte (default)
------------------------------------------------------------
[400] local 192.168.1.101 port 50826 connected with 192.168.1.102 port 5001
[156] local 192.168.1.101 port 5001 connected with 192.168.1.102 port 49125
[ ID] Interval Transfer Bandwidth
[400] 0.0-10.0 sec 49.6 MBytes 41.6 Mbits/sec
[156] 0.0-10.0 sec 311 MBytes 261 Mbits/sec 0.085 ms 463488/685436 (68%)
[156] 0.0-10.0 sec 1 datagrams received out-of-order
[400] Server Report:
[400] 0.0-10.0 sec 49.6 MBytes 41.6 Mbits/sec 0.472 ms 0/35365 (0%)
[400] Sent 35365 datagrams |
|
|
| Back to top |
|
|
Mad Merlin
Veteran
Joined: 09 May 2005
Posts: 1134
|
|
| Back to top |
|
|
urgh
n00b
Joined: 16 Apr 2012
Posts: 5
|
| Posted: Thu Apr 19, 2012 10:50 pm Post subject: |
|
|
I'll try some network tuning and check if I get better throughput.
If it fails, I will ask in network forum, but at least I know it's not disk related.
Thanks for help  |
|
| Back to top |
|
|
|
Kernel & Hardware
