Extremely slow USB writes [solved]

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Hi Guys/Gals, For a while now I've been struggling with very poor USB write speeds. It's borderline unusable

My system on has XHCI:

NeddySeagoon · Posted: Sun Feb 21, 2016 1:29 pm Post subject:

cdstealer,

Some USB3 root hubs don't like to mix USB2 and USB3 devices.
If you connect a USB2 device first, it will not operate at USB3 speeds when you connect a USB3 device later.

Your lsusb suggests that you only have USB3 root hubs.
Check with lspci. If thats true, do not build the USB1 and USB2 drivers for your kernel, as the XHCI driver will take care of everything.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Hi Neddy, I only have the XHCI driver compiled as that is the only USB chipset on the board. I'll keep digging as there are no errors reported anywhere.

ta
_________________
# touch it
touch: cannot touch `it': Permission denied

NeddySeagoon · Posted: Sun Feb 21, 2016 4:10 pm Post subject:

cdstealer,

Boot with no USB devices connected.
Connect your USB3 pen drive and speed test that.
Then add in your USB2 pen drive and speed test the USB3 again then the USB2.

Can you format the USB3 device?
You may be running into slowdowns caused by waiting for blocks to erase.
Formatting will offer to trim the whole device, so that there is no erase wait, providing the device supports trim.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Hi Neddy,
Yep, already done all that. I even left the USB3 drive doing a full format which took several hours with no improvement

Although I've just tried something else. I had a USB3 2.5" enclosure buried away. So I put a mechanical HDD in and that doesn't appear to suffer. So you maybe correct in that the issue is with the flash drives themselves. Time for some more fault finding.

Thank you very much for your help so far, it's very appreciated

spectromas · n00b Joined: 13 Jul 2014 Posts: 50

Does the target filesystem make any difference? I have a ongoing issue writing to ntfs taking forever on usb2 and causing very bad performance while the writing is taking place, with xfs or ext3/4 it is fine.

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Hi spectromas, I tried both VFAT and EXT4. EXT4 it appears in terminal to be transferring at the correct speed, but actually isn't and with VFAT, the terminal output stalls after each file until it has been written to the flash drive.

It's entirely possible that I've missed an option out of the kernel as I've just transferred a 144Mb file on my work laptop which transferred @22Mbs. I'll compare the configs when I get home and report back.
_________________
# touch it
touch: cannot touch `it': Permission denied

spectromas · n00b Joined: 13 Jul 2014 Posts: 50

Please do post the results here if you find anything that makes a difference, maybe I've missed something too.

NeddySeagoon · Posted: Mon Feb 22, 2016 5:59 pm Post subject:

spectromas,

ntfs on Linux is a FUSE filesysem. There is a speed penalty for that.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

spectromas · n00b Joined: 13 Jul 2014 Posts: 50

NeddySeagoon, that is what I have come to accept but I have a feeling something isn't right somewhere, I don't think the system should be come unusable when a file is being transferred. I've been through lots of troubleshooting and could never narrow it down much. However, tonight I tried out manjaro (as much as I dislike it) on another hard drive because I know it comes with a kernel with bfs and bfq enabled. Sure enough file transfer to ntfs was smooth as silk, drastically different from what I have at the moment. Now the different factors I can think of include, kernel config, bfs, bfq, different hard drive, ext3 instead of xfs, systemd. I've tried bfs and bfq together and each separately and hadn't previously noticed much difference but it could be that I suppose. The others could all be contenders as far as I can see, thinking it could be my drive though as not many other people seem to share the same problem. Transfer speed was still quite slow but still faster. Ejecting/unmounting was also much faster, sometimes it takes my system 10mins or so after a file of a few gbs, it took about 1min in manjaro.

Not intending to hijack the thread but this is certainly food for thought for me, just being able to do stuff while transferring files and it not taking 30mins to transfer and unmount a few gb sized file is revelatory right now.

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Had time to test properly on my work laptop today. It only has USB2, but it would prove a point.

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Hi Guys, I'm wrapping this one up. After many kernel compiles and testing different OSes and USB drives. I can categorically confirm that my 64Gb Corsair USB3.0 drive is not fit for purpose. Time to go research a full speed replacement. :roll:

Here are my benchmark results.

http://cdstealer.com/images/usb_benchmark.jpg

Thanks again.
_________________
# touch it
touch: cannot touch `it': Permission denied

P.Kosunen · Guru Joined: 21 Nov 2005 Posts: 309 Location: Finland

https://www.sandisk.com/home/usb-flash/extreme-usb

SanDisk Extreme USB has good write performance, even random write.

cdstealer · Guru Joined: 30 Oct 2005 Posts: 431 Location: Leeds

Cheers P.Kosunen I'll go check it out

_________________
# touch it
touch: cannot touch `it': Permission denied