View previous topic :: View next topic |
Author |
Message |
Diovanti n00b
Joined: 27 Feb 2003 Posts: 63
|
Posted: Thu Apr 14, 2011 8:34 pm Post subject: 3TB USB HD is showing up as 801GB |
|
|
I just got a 3GB Hitachi HD and put it into a USB external hard drive enclosure. The problem is linux cannot see its full capicity. The drive in the enclosure has been tested with a windows 7 machine connecting with esata and it works fine under windows.
Code: |
Linux nirvana 2.6.36.2 #1 SMP Thu Mar 3 19:37:29 Local time zone must be set--see zic m x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux
[610730.148026] scsi 14:0:0:0: Direct-Access Hitachi HDS5C3030ALA630 PQ: 0 ANSI: 2 CCS
[610730.148831] sd 14:0:0:0: Attached scsi generic sg4 type 0
[610730.149306] sd 14:0:0:0: [sdd] 1565565872 512-byte logical blocks: (801 GB/746 GiB)
[610730.151917] sd 14:0:0:0: [sdd] Write Protect is off
[610730.151924] sd 14:0:0:0: [sdd] Mode Sense: 00 38 00 00
[610730.151928] sd 14:0:0:0: [sdd] Assuming drive cache: write through
[610730.154882] sd 14:0:0:0: [sdd] Assuming drive cache: write through
[610730.177121] sdd: unknown partition table
[610730.180915] sd 14:0:0:0: [sdd] Assuming drive cache: write through
[610730.180923] sd 14:0:0:0: [sdd] Attached SCSI disk
|
The 'sdd: unknown partition table' is a result of what happens when I partition the drive in windows and the total size of partition exceeds' 800gb. I can clear the disklabel using parted and create a new GPT label that linux will read, but I cant make it more than 800GB. It will say the 'Error: The location is outside of the device /dev/sdd'
Any ideas how to get this to work. The external drive is going to be hooked up to an ARM machine running debian in the future so I hope any workaround can be applied to that as well. |
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Thu Apr 14, 2011 10:24 pm Post subject: |
|
|
cat /usr/src/linux/.config | grep LBDAF ? |
|
Back to top |
|
|
Jon Wilder Tux's lil' helper
Joined: 04 Apr 2011 Posts: 112 Location: Fresno CA
|
Posted: Fri Apr 15, 2011 12:36 am Post subject: |
|
|
Diovanti wrote: | I just got a 3GB Hitachi HD |
Hopefully you meant a 3TB drive.
Under "Enable the block layer" in the kernel config menu, do you have "Support for large (2TB+) block devices and files" compiled in?
Code: |
Top level menu
Enable the block layer------>
[*]Support for large (2TB+) block devices and files"
|
If not, you'll need to enable that in the kernel menu, recompile the kernel, then reboot to that kernel. |
|
Back to top |
|
|
Diovanti n00b
Joined: 27 Feb 2003 Posts: 63
|
Posted: Fri Apr 15, 2011 8:36 am Post subject: |
|
|
Jon Wilder wrote: | Diovanti wrote: | I just got a 3GB Hitachi HD |
Hopefully you meant a 3TB drive.
Under "Enable the block layer" in the kernel config menu, do you have "Support for large (2TB+) block devices and files" compiled in?
Code: |
Top level menu
Enable the block layer------>
[*]Support for large (2TB+) block devices and files"
|
If not, you'll need to enable that in the kernel menu, recompile the kernel, then reboot to that kernel. |
Its a 3TB HD Here's a screenshot from my menuconfig 2.6.36.2 vanilla (I don't see the option for large block listed, grep showes nothing as well):
Code: |
.config - Linux Kernel v2.6.36.2 Configuration
──────────────────────────────────────────────────────────────────────────────
┌──────────────────────── Enable the block layer ─────────────────────────┐
│ Arrow keys navigate the menu. <Enter> selects submenus --->. │
│ Highlighted letters are hotkeys. Pressing <Y> includes, <N> excludes, │
│ <M> modularizes features. Press <Esc><Esc> to exit, <?> for Help, </> │
│ for Search. Legend: [*] built-in [ ] excluded <M> module < > │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ --- Enable the block layer │ │
│ │ [*] Block layer SG support v4 │ │
│ │ [ ] Block layer data integrity support │ │
│ │ IO Schedulers ---> │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────────┤
│ <Select> < Exit > < Help > │
└─────────────────────────────────────────────────────────────────────────┘
|
P.S. If the drive is hooked up to windows via usb without being partitioned it shows up as ~800GB. When using esata it shows up as 3TB. However after partitioning it in windows for the full 3TB it will then show up as the full 3TB using usb in windows as well. Unfortunately my linux systems don't have esata so I need to use usb |
|
Back to top |
|
|
madchaz l33t
Joined: 01 Jul 2003 Posts: 993 Location: Quebec, Canada
|
Posted: Fri Apr 15, 2011 1:11 pm Post subject: |
|
|
screengrabs of menuconfig are pretty useless. Try the cat/grep command krinn posted to see if you have the option. It is required to access disks that are that large. _________________ Someone asked me once if I suffered from mental illness. I told him I enjoyed every second of it.
www.madchaz.com A small candle of a website. As my lab specs on it. |
|
Back to top |
|
|
Anon-E-moose Watchman
Joined: 23 May 2008 Posts: 6098 Location: Dallas area
|
Posted: Fri Apr 15, 2011 2:47 pm Post subject: |
|
|
You won't find the large block option on 64 bit systems
Code: | config LBDAF
bool "Support for large (2TB+) block devices and files"
depends on !64BIT |
The bios on the linux box may also be limiting how much of the disk you see. _________________ PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
Back to top |
|
|
Diovanti n00b
Joined: 27 Feb 2003 Posts: 63
|
Posted: Fri Apr 15, 2011 2:57 pm Post subject: |
|
|
Anon-E-moose wrote: | You won't find the large block option on 64 bit systems
Code: | config LBDAF
bool "Support for large (2TB+) block devices and files"
depends on !64BIT |
The bios on the linux box may also be limiting how much of the disk you see. |
Yeah, I'm on 64bit. Is there a way to tell linux the disk geometry manually without going with what the bios says? |
|
Back to top |
|
|
Anon-E-moose Watchman
Joined: 23 May 2008 Posts: 6098 Location: Dallas area
|
Posted: Fri Apr 15, 2011 3:47 pm Post subject: |
|
|
Diovanti wrote: | Is there a way to tell linux the disk geometry manually without going with what the bios says? |
Don't know.
I would google whatever the motherboard is on the linux box...with 3tb drives...and see what shows up.
You could also check with the mb manufacturer and see if they have a bios update that addresses large disks. _________________ PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Fri Apr 15, 2011 7:35 pm Post subject: |
|
|
If it works without the USB in the way, then the USB adaptor is junk. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54244 Location: 56N 3W
|
Posted: Fri Apr 15, 2011 9:26 pm Post subject: |
|
|
Diovanti,
Code: | [610730.149306] sd 14:0:0:0: [sdd] 1565565872 512-byte logical blocks: (801 GB/746 GiB) |
Thats wrong. 3Tb drives have a 4k native sector size. Something somewhere is assuming 512b sectors.
Take care when you partition the drive, partitions are aligned on 4k boundarys. If not, the drive will fake 512b sectors by doing read/modify/writes which is 20 to 30 times slower than properly aligned writes. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
chithanh Developer
Joined: 05 Aug 2006 Posts: 2158 Location: Berlin, Germany
|
Posted: Sat Apr 16, 2011 8:16 am Post subject: |
|
|
Almost all drives will present 512 byte logical sectors to the OS. Very few exceptions exist (e.g. Samsung S1 Mini) which use 4k logical sectors.
You could try to upgrade your kernel. |
|
Back to top |
|
|
Diovanti n00b
Joined: 27 Feb 2003 Posts: 63
|
Posted: Sun Apr 17, 2011 8:15 am Post subject: |
|
|
chithanh wrote: | Almost all drives will present 512 byte logical sectors to the OS. Very few exceptions exist (e.g. Samsung S1 Mini) which use 4k logical sectors.
You could try to upgrade your kernel. |
Upgraded kernel to 2.6.38.2 Still shows bogus capacity.
Ant P. wrote: | If it works without the USB in the way, then the USB adaptor is junk. |
I tried hooking up the drive to a more modern system directly using SATA. I booted a Ubuntu 10.10 64bit live cd and it recognized the full capacity. I partitioned it with gparted starting from sector 2048 (so it wouldn't complain about not being aligned) to the end of the disk. I then formatted it NTFS and copied some files to the disk. It worked.
Then I umounted the disk and attached it to the same computer as usb. It was detected as 801GB again and dmesg said 'invalid partition table'. So, it seems there is a problem with my enclosure : http://www.newegg.com/Product/Product.aspx?Item=N82E16817182179
However, Windows 7 detects my ntfs partition correctly when using USB Does anyone have any idea what kind of magic voodoo it does to detected the drive correctly and if that can be applied to linux? (I plan to use this disk on Linux and Windows so I need it to be NTFS.) |
|
Back to top |
|
|
chithanh Developer
Joined: 05 Aug 2006 Posts: 2158 Location: Berlin, Germany
|
Posted: Sun Apr 17, 2011 8:56 am Post subject: |
|
|
It appears to me that this is a kernel problem. If the enclosure is a recent model, you can try to toggle USB Attached SCSI support in your kernel, maybe that will make a difference.And you could report a bug on https://bugzilla.kernel.org/ |
|
Back to top |
|
|
Diovanti n00b
Joined: 27 Feb 2003 Posts: 63
|
Posted: Sun Apr 17, 2011 9:43 am Post subject: |
|
|
Also one thing I should be a little more clear on: Windows 7 will show the drive's true capacity using usb only when it has already been partitioned previously using a esata/sata connection. If the drive is blank, it will show up to windows as ~800 GB
Maybe the drive's firmware is reporting 800GB? Perhaps for some compatibility reasons for older BIOSes? There's no jumpers on this thing, though. The drives model is: Hitachi HDS5C3030ALA630. I didn't see anyting about this on the manufacturers website. I found another post where someone was having a smilar model Hatachi drive showing up as 801GB as well: http://hardforum.com/showthread.php?t=1573971
chithanh wrote: | It appears to me that this is a kernel problem. If the enclosure is a recent model, you can try to toggle USB Attached SCSI support in your kernel, maybe that will make a difference.And you could report a bug on https://bugzilla.kernel.org/ |
I just made the uas modules and modprobed it. Still dosen't work.
[/code]
[ 5348.811223] usbcore: registered new interface driver uas
[ 5395.851973] hub 1-0:1.0: state 7 ports 10 chg 0000 evt 0200
[ 5395.851987] ehci_hcd 0000:00:02.1: GetStatus port:9 status 001002 0 ACK POWER sig=se0 CSC
[ 5395.851996] hub 1-0:1.0: port 9, status 0100, change 0001, 12 Mb/s
[ 5395.852000] usb 1-9: USB disconnect, address 3
[ 5395.852010] usb 1-9: unregistering device
[ 5395.852013] usb 1-9: unregistering interface 1-9:1.0
[ 5395.854601] usb 1-9: usb_disable_device nuking all URBs
[ 5395.958015] hub 1-0:1.0: debounce: port 9: total 100ms stable 100ms status 0x100
[ 5414.061991] hub 1-0:1.0: state 7 ports 10 chg 0000 evt 0200
[ 5414.062004] ehci_hcd 0000:00:02.1: GetStatus port:9 status 001803 0 ACK POWER sig=j CSC CONNECT
[ 5414.062021] hub 1-0:1.0: port 9, status 0501, change 0001, 480 Mb/s
[ 5414.166006] hub 1-0:1.0: debounce: port 9: total 100ms stable 100ms status 0x501
[ 5414.217214] ehci_hcd 0000:00:02.1: port 9 high speed
[ 5414.217219] ehci_hcd 0000:00:02.1: GetStatus port:9 status 001005 0 ACK POWER sig=se0 PE CONNECT
[ 5414.268008] usb 1-9: new high speed USB device using ehci_hcd and address 4
[ 5414.319224] ehci_hcd 0000:00:02.1: port 9 high speed
[ 5414.319228] ehci_hcd 0000:00:02.1: GetStatus port:9 status 001005 0 ACK POWER sig=se0 PE CONNECT
[ 5414.383446] usb 1-9: default language 0x0409
[ 5414.384322] usb 1-9: udev 4, busnum 1, minor = 3
[ 5414.384324] usb 1-9: New USB device found, idVendor=152d, idProduct=2336
[ 5414.384327] usb 1-9: New USB device strings: Mfr=1, Product=2, SerialNumber=5
[ 5414.384330] usb 1-9: Product: JM20336 SATA, USB Combo
[ 5414.384333] usb 1-9: Manufacturer: JMicron
[ 5414.384335] usb 1-9: SerialNumber: 13231874C888
[ 5414.384540] usb 1-9: usb_probe_device
[ 5414.384545] usb 1-9: configuration #1 chosen from 1 choice
[ 5414.385079] usb 1-9: adding 1-9:1.0 (config #1, interface 0)
[ 5414.385620] usb-storage 1-9:1.0: usb_probe_interface
[ 5414.385628] usb-storage 1-9:1.0: usb_probe_interface - got id
[ 5414.386473] scsi8 : usb-storage 1-9:1.0
[ 5414.386687] drivers/usb/core/inode.c: creating file '004'
[ 5415.388845] scsi 8:0:0:0: Direct-Access Hitachi HDS5C3030ALA630 PQ: 0 ANSI: 2 CCS
[ 5415.389606] sd 8:0:0:0: Attached scsi generic sg4 type 0
[ 5415.390076] sd 8:0:0:0: [sdd] 1565565872 512-byte logical blocks: (801 GB/746 GiB)
[ 5415.392739] sd 8:0:0:0: [sdd] Write Protect is off
[ 5415.392746] sd 8:0:0:0: [sdd] Mode Sense: 00 38 00 00
[ 5415.393323] sd 8:0:0:0: [sdd] Asking for cache data failed
[ 5415.393327] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[ 5415.397319] sd 8:0:0:0: [sdd] Asking for cache data failed
[ 5415.397323] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[ 5415.437740] sdd:
[ 5415.443460] sd 8:0:0:0: [sdd] Asking for cache data failed
[ 5415.443466] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[ 5415.443470] sd 8:0:0:0: [sdd] Attached SCSI disk
nirvana linux-2.6.38.2 # parted /dev/sdd
GNU Parted 2.3
Using /dev/sdd
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Error: Can't have a partition outside the disk!
(parted)
[/code] |
|
Back to top |
|
|
chithanh Developer
Joined: 05 Aug 2006 Posts: 2158 Location: Berlin, Germany
|
Posted: Sun Apr 17, 2011 9:48 am Post subject: |
|
|
Ok in that case I take that back and it could be a problem of the USB enclosure. Did you try to contact the manufacturer? If it doesn't work properly on Windows, then they can hardly refuse support. |
|
Back to top |
|
|
Anon-E-moose Watchman
Joined: 23 May 2008 Posts: 6098 Location: Dallas area
|
Posted: Sun Apr 17, 2011 10:52 am Post subject: |
|
|
A lot of USB external enclosures, mention that they are good up to 2TB.
It probably requires a firmware upgrade (if offered) for the enclosure.
Something to keep in mind if one wants to use 2TB+ disks in them.
Note: when disks get over 2TB then many systems have problems with the MBR.
GPT (GUID Partition Table) will be the next big thing for VLD (Very Large Disks)
but very few motherboard and enclosure manufacturers support that yet.
Edit to add: from this link http://ubuntuforums.org/archive/index.php/t-1657961.html
Quote: | Unfortunately, my old RX35-AT-SC enclosures (I have two) are not compatible with 3TB drives. |
_________________ PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Last edited by Anon-E-moose on Sun Apr 17, 2011 9:03 pm; edited 1 time in total |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54244 Location: 56N 3W
|
Posted: Sun Apr 17, 2011 11:10 am Post subject: |
|
|
The enclosure should not care about data on the drive, including filesystem metadata.
The OS handles the filesystem, the drive gets commands at the block level.
800G is a strange size to get reported too. Its not the 137G limit of early LBA so the enclosure is clearly seeing 48 bit LBA, or it couldn't report the drive size as 800G. The drives response to the identify command is a 512b data block ... just like any block read off the disk.
The enclosure shouldn't care about that either.
As I said earlier, a 3T drive will use 4k physical sectors, 512b blocks will be faked. That should not be an issue for the enclosure. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Anon-E-moose Watchman
Joined: 23 May 2008 Posts: 6098 Location: Dallas area
|
Posted: Sun Apr 17, 2011 11:54 am Post subject: |
|
|
NeddySeagoon wrote: | The enclosure should not care about data on the drive, |
It may not care about the data, per se, but if it doesn't understand the geometry of the disk,
then there will be a problem, and the electronics/interface/firmware on the enclosure is what handles that.
PC's had that problem long ago where they couldn't handle 8+ gig disks. _________________ PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54244 Location: 56N 3W
|
Posted: Sun Apr 17, 2011 2:02 pm Post subject: |
|
|
Anon-E-moose,
Disk Geometry has been a fiction since drives reached 4G. Everything is Logical Block Addressing (LBA) now.
Geometry is only used by old tools and operating systems and as you say, the most you can address with CHS is about 8G _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Sun Apr 17, 2011 2:22 pm Post subject: |
|
|
Can you actually put >2TB of data on the drive in windows through USB?
The enclosure has to support it (the usb<->sata translation unfortunately isn't "dumb" enough to ignore the size), otherwise it shouldn't work at all.
If there's any way to give your Linux box eSATA capability, do that and forget about USB... it's much faster than USB2 too... |
|
Back to top |
|
|
Anon-E-moose Watchman
Joined: 23 May 2008 Posts: 6098 Location: Dallas area
|
Posted: Sun Apr 17, 2011 2:28 pm Post subject: |
|
|
LBA still has it's problems and limits.
So no we haven't gotten away from geometry, we've just expanded it out some
There are many articles mentioning these problems, here's part of one
Quote: | moving to a capacity greater than 2.1 TB requires more work than merely upping the areal density. In fact, most PCs just aren't built to cope with hard drive capacities beyond that limit thanks to the original logical block addressing (LBA) standard set by Microsoft and IBM twenty years ago.
We've heard this before song and dance before: the limited LBA was originally designed for DOS, and only allotted 512-byte sectors for each sector, locking capacities to a 2.1 TB limit. According to Craig, at the time no one imagined that technology would allow for capacities over 2.1 TB when the LBA standard was developed in 1980.
With that said, Long LBA addressing will need to be applied to get around the capacity lock, requiring 64-bit versions of Windows 7 and Windows Vista, and modified versions of Linux. As for Windows XP, Long LBA addressing leaves the older OS out of the picture. Craig said that in-house tests have shown that only 990 MB of a 3 TB drive is available in XP, with the remaining 2.1 TB literally unseen by the OS.
But the current LBA isn't the only hurdle. Craig said that a new GUID partition table (GPT) will need to be released, as current master boot record partitions are locked to 2.1 TB. GPT and Long LBA addressing are now part of the new UEFI system, however this new BIOS replacement hasn't become standard, and is only used in a minimum number of motherboards. As it stands now, a 3 TB drive is pointless until the industry is overhauled, including motherboards, RAID controllers, drivers, and operating systems. |
Anyway enough of my prattling.
Be well my friend _________________ PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
Back to top |
|
|
DONAHUE Watchman
Joined: 09 Dec 2006 Posts: 7651 Location: Goose Creek SC
|
|
Back to top |
|
|
Diovanti n00b
Joined: 27 Feb 2003 Posts: 63
|
Posted: Sun Apr 17, 2011 8:56 pm Post subject: |
|
|
Thanks for all the replies. The enclosure just isn't compatible with a 3TB drive. I've ordered one specifically that offers 3TB support.
While I could add e-sata to my gentoo box, this drive is going to be soon hooked up to a plug computer (Guruplug) and all it has are two USB ports. |
|
Back to top |
|
|
mbjr Guru
Joined: 17 Jan 2004 Posts: 531 Location: Budapest/Hungary
|
Posted: Mon Dec 23, 2013 7:13 pm Post subject: |
|
|
*bump*
Did the new enclosure solve it?
I have 3 different enclosures from different manufacturers, an older one back from a year ago and and one I have just purchased to solve this particular problem :> _________________ mb |
|
Back to top |
|
|
|