HORRIBLE Disk to Disk copy/move performance

Message

zakl · Post by **zakl** » Fri May 23, 2003 9:17 pm

My hardware setup is as follows...

Athlon XP 1700+
EpoX 8KHA+ (via KT266a chipset)
768MB PC2100 (kingston)
IDE Channel 1 (using ATA133 cable):
Master - IBM 40GB ATA100
Slave - Plextor 40/12/40A Burner
IDE Channel 2 (using ATA100 cable):
Master - Western Digital WD1200BB 120GB ATA100
Slave - Sony 16x DVD

The WD 120 is formatted at max capacity as FAT32, while the IBM is split in half... W2k Pro takes first half of drive, followed by /boot par, root par, and swap par. Gentoo's half gives swap 1GB and the rest goes to root par.

This is the problem, when I copy/move from drive to drive, I average 2-3MB/s! I started a 4GB move, and it started at 27MB/s for about 6 seconds then dropped to 3MB/s. Every once in a while it will bounce up to 6MB/s, but that's it. It takes me almost a minute and a half to move a 200MB file. I know it's not my hardware setup, because in W2K, I can move a 200MB file between the drives in 10-15 seconds, which is how it should be.

hdparm -Tt reports:

root@Logical yoyo # hdparm -Tt /dev/hdc

/dev/hdc:
Timing buffer-cache reads: 128 MB in 0.44 seconds =290.91 MB/sec
Timing buffered disk reads: 64 MB in 1.38 seconds = 46.38 MB/sec
root@Logical yoyo # hdparm -Tt /dev/hda

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.44 seconds =290.91 MB/sec
Timing buffered disk reads: 64 MB in 1.83 seconds = 34.97 MB/sec

hdparm -d shows same thing on both drives:
using_dma = 1 (on)

I can post my hdparm -d results if requested.

Thanks,

Zak

taskara · Post by **taskara** » Fri May 23, 2003 11:24 pm

hmm.. as a test, try disconnecting your dvd and burner and have JUST the hard drives connected.

let me know if it improves at all.

amne · Post by **amne** » Sat May 24, 2003 9:06 am

which kernel are you using? i have the same motherboard and there seemed to be some problems with the chipset, i think. in 2.4.18 copying large files sometimes even froze the system for a few seconds and the copying didn't go on in that time as well, 2.4.19 was better and 2.4.20 (all vanilla) works pretty fine for me.

zakl · Post by **zakl** » Fri Jun 06, 2003 11:18 pm

I'm using ck-sources kernel 2.4.20-ck6. I haven't got a chance to disconnect my drives, but I will ASAP.

Thanks for the help,

Zak

zakl · Post by **zakl** » Sun Jun 29, 2003 8:50 pm

Could it be possible that my round ATA133 cable is causing this slowdown? I have a 24" ATA133 cable on channel 0.

I've always read in the past that IDE cables have a maximum length of 18", and anything longer causes attenuation. Also, my IBM is at the very end of the cable, which means it's 24" from the mother board. This is something I haven't thought about till now.

Is it possible that the length of that cable is causeing my problems?

Zak

flokno · Post by **flokno** » Mon Jun 30, 2003 12:29 am

i have had the same or a similar problem.
have a look here: http://forums.gentoo.org/viewtopic.php?p=239841#239841
after disabling this validation bit setting some weeks ago (now i don't think that it is necessary) and carefully disabling all not necessary ide support in ide config things are working without a problem now. have a look at every help-text available there and in case of doubt follow recommendations.
i can post my kernel config if necessary though i only use vanilla-sources-2.4.21.

-- flo

zakl · Post by **zakl** » Mon Jun 30, 2003 1:29 pm

Thanks for the tip. It was not enabled, so I enabled it and recompiled my kernel. I will reboot to test it as soon as OpenOffice 1.1 finishes compiling, and post the results here.

And man does OpenOffice take a long time to compile! It's been going for over 12 hours now on an Athlon XP 1700+, and used up 2GB of disk space.

zakl · Post by **zakl** » Mon Jun 30, 2003 2:23 pm

Nope, that kernel option made no difference. The thing that gets me is it's only when I try to copy between drives in the machine. I have two other machines on the LAN, and I can copy from them over the network @ 10MB/s to either drive. But when if I copy between the two drives it's stuck around 3-4MB. I'm going to try what taskara requested, as well as taking out the 24" IDE cable.

I'll post my results here.

Zak

zakl · Post by **zakl** » Mon Jun 30, 2003 2:46 pm

Ok, I disconnected both of my CD drives, and performed a copy test. Now drive to drive copy is between 10-30MB/s. Why would my CD drives be causing conflict? I did notice however, that hdparm gives me an error whenever I try to use it with my Plextor burner. Says something about not being compatible, or hdparm does not work with drive. I'll get the exact error when I hook my drives back up. Could that be the problem?

I should have known based on the fact that windows didn't have the same problem, that it was not my IDE cable length.

What can I do? I move LOTS of files between my hard drives, and 3MB/s is just not gonna cut it. And operating my system with no CD drives isn't gonna cut it either. Please help me out.

zakl · Post by **zakl** » Mon Jun 30, 2003 3:08 pm

Ok, this is tripped out, but after reconnecting my CD drives, it's still copying at 10-30MB/s now. No complaining here.

The error hdparm was giving me for my plextor is as follows.

root@logical banji # hdparm -d /dev/cdroms/cdrom1
/dev/cdroms/cdrom1 not supported by hdparm

But if performance stays the same, it's all good.

Taskara, thanks for the test, I have no clue what was wrong, but reconnecting the drives seems to have fixed it.

Zak

taskara · Post by **taskara** » Mon Jun 30, 2003 11:51 pm

groovy

MasterX · Post by **MasterX** » Tue Jul 01, 2003 6:29 am

Sorry to bring this up, but I did not quite understand how you fixed your problem.
My problem is that when I am emerging files or when I am copying/moving the system almost freezes. The pointer does not move, I can not change desktops etc.
The hdparm -d gave
using_dma = 0 (off)

how can I change it to 4 (This is what I have in the Bios)
Can I do the same thing for the CD driver/Dvd

taskara · Post by **taskara** » Tue Jul 01, 2003 7:31 am

you won't change it for a burner or dvd if you're using scsi emulation, and for your hdd you don't want to change

using_dma = 0 (off)

to 4, just to 1 but you want to change the access mode to 4 (ata66)

if so then this if what you want to run on your device

Code: Select all

hdparm -c1 -d1 -m16 -X68 -k1 /dev/hda

then check the output again and test with

Code: Select all

hdparm -tT /dev/hda

taskara · Post by **taskara** » Tue Jul 01, 2003 7:32 am

btw if you're using ata100, then subsitute -X69

MasterX · Post by **MasterX** » Tue Jul 01, 2003 3:46 pm

Taskara, thanks for the info.
This is what I got

/dev/hdb:
setting 32-bit IO_support flag to 1
setting multcount to 16
setting using_dma to 1 (on)
HDIO_SET_DMA failed: Operation not permitted
setting keep_settings to 1 (on)
setting xfermode to 69 (UltraDMA mode5)
HDIO_DRIVE_CMD(setxfermode) failed: Input/output error
multcount = 16 (on)
IO_support = 1 (32-bit)
using_dma = 0 (off)
keepsettings = 1 (on)

So, I was not able to change it

MasterX · Post by **MasterX** » Tue Jul 01, 2003 6:05 pm

If I need to umount /dev/hdb in order hdprm to work, how can I do that since I am using Linux and /dev/hdb has to be mounted

Thanks

taskara · Post by **taskara** » Tue Jul 01, 2003 10:44 pm

if you can't enable dma, it sounds like the driver for your ide controller is not compiled into your kernel, or you don't have dma for hard disks enabled.

check your kernel config again.

what is your ide controller?

MasterX · Post by **MasterX** » Tue Jul 01, 2003 11:30 pm

I looked at the config file for the kernel
In the section IDE,ATA and Atari support, the following options are checked
Generic PCI bus-master DMA support
Use PCI-DMA by deafult when available

Is this what you were talking about?
How can I find the Ide-controler?

Thanks

taskara · Post by **taskara** » Tue Jul 01, 2003 11:50 pm

what is the model and brand of your mainboard?

MasterX · Post by **MasterX** » Tue Jul 01, 2003 11:59 pm

Asus P4Pe with intel 845 chipset

taskara · Post by **taskara** » Wed Jul 02, 2003 12:19 am

if you plugged your hdd into the main ide channel, do you have

Code: Select all

IntelPIIXn chipsets support

support compiled in to your kernel? and if so, as a module or directly in?

MasterX · Post by **MasterX** » Wed Jul 02, 2003 2:02 am

No, it is included, I will built in and let you know

MasterX · Post by **MasterX** » Wed Jul 02, 2003 2:07 am

I can not find the PIIXn Tuning support. Is that a problem?

MasterX · Post by **MasterX** » Wed Jul 02, 2003 2:30 am

Thanks a lot Taskara. Now using_dma is on

taskara · Post by **taskara** » Wed Jul 02, 2003 2:35 am

you shoudl not need tuning

just the other built directly in