Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Any "Advanced Format Drive" Pitfalls?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
Caligatio
n00b
n00b


Joined: 13 Feb 2010
Posts: 48

PostPosted: Fri May 07, 2010 8:41 pm    Post subject: Any "Advanced Format Drive" Pitfalls? Reply with quote

I have managed to get Gentoo running on both my laptop and desktop successfully (woo!) and am feeling confident enough to make myself an internal file/ssh/svn/git/web server. I've started ordering computer parts and just got my 2 kernel RAID-1 destined hard drives.

I ended up purchasing the Western Digital 1.5 Green drives (WD15EARS) which came with this lovely sticker warning that if you use XP, you need to run some utility because they're Advanced Format Drives. I did some research and this is a new shiny 4 KiB sector drive. Some additional links of interest:

http://thread.gmane.org/gmane.linux.utilities.util-linux-ng/2926
http://community.wdc.com/t5/Desktop/Problem-with-WD-Advanced-Format-drive-in-LINUX-WD15EARS/m-p/6395

The first link seems to say that as long as you're using 2.6.32 or 2.6.33 you shouldn't have to worry about it. Can anyone confirm this? Everyone references "parted" and not "fdisk."


Thanks!
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Fri May 07, 2010 9:05 pm    Post subject: Reply with quote

The pitfall with the WD15EARS is that it does not report its nature to the OS, so it looks like a normal 512byte sector disk. hence the partitioning tools and mkfs do not automatically use 4k alignment - you have to take care of that manually. You have to verify that every layer you put on the disk (in my case GPT -> cryptsetup/LUKS -> LVM -> XFS/reiserfs/ext23) uses 4k sectors. Not just during install but every time you change something too. The kernel does not help you there at all yet (I even tried 2.6.34-rc6), I guess their improved 4k support means on a block driver level and probably also only for disks that report themselves as 4k disks too...
Back to top
View user's profile Send private message
Caligatio
n00b
n00b


Joined: 13 Feb 2010
Posts: 48

PostPosted: Fri May 07, 2010 9:19 pm    Post subject: Reply with quote

I'm going to have to show my noobishness here:

Assuming I'm doing a kernel RAID-1 and EXT 2/4, what steps am I going to have to make sure I don't mess up?

How do I make fdisk work sectors and not cylinders?
EDIT: Looks like -u might be what I want
I don't think I need/want cryptsetup/LUKS
Is there anything during the RAID-1 setup? I'm looking over the handbook and it doesn't seem to mention sectors. I also don't think I need LVM.
Is there anything to be mindful of during formatting with EXT2 and EXT4?
EDIT: -b 4096 I think will do it

I really appreciate any help!

EDIT: Found this link: https://forums.gentoo.org/viewtopic-t-815280.html but it doesn't seem to really help.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Fri May 07, 2010 9:58 pm    Post subject: Reply with quote

just for my curiosity, are those disks really cheaper ? because except for price i don't get the point of using green disks that are slower than "normal" disks (not sure i read some users saying that, and it seems also logic, slower disk = less energy...), specially to build a raid-1.
I mean if you need speed, why choose slow disks?

And i don't take the green consideration in case, how can someone buy "green" disks to save the world and using gentoo that might be (i says might, but i think IS) the worst os to save the planet

Must have something i didn't get or some weird choice there.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Fri May 07, 2010 10:08 pm    Post subject: Reply with quote

The WD15EARS when I bought it, was the cheapest disk by WD (50€/TB whereas the cheapest blue is already 70€/TB). Pitfalls aside, it also fits my needs perfectly - regarding speed, my bottleneck is encryption anyways, but I'm not unhappy with the drive's speed, quite the opposite... I have another older 1TB green as external backup disk, which is slower...

parted align doesn't really work... at least it didn't for me for this drive.

I'm not sure whether you have to do anything special for RAID, since I'm not using RAID on the drive, sorry... for mkfs, for most filesystems you can specify the block size with an option (read the manpage), for partitioning, you can print the partition layout in 512b sectors and make sure that every partition starts at a 4k boundary.
Back to top
View user's profile Send private message
Caligatio
n00b
n00b


Joined: 13 Feb 2010
Posts: 48

PostPosted: Sat May 08, 2010 2:24 am    Post subject: Reply with quote

krinn wrote:
just for my curiosity, are those disks really cheaper ? because except for price i don't get the point of using green disks that are slower than "normal" disks (not sure i read some users saying that, and it seems also logic, slower disk = less energy...), specially to build a raid-1.
I mean if you need speed, why choose slow disks?

And i don't take the green consideration in case, how can someone buy "green" disks to save the world and using gentoo that might be (i says might, but i think IS) the worst os to save the planet

Must have something i didn't get or some weird choice there.


The disks are relatively cheap, $90 for 1.5 TB but I was honestly going for lower power. Also I'm just streaming media to about 3-5 machines on my home network.. nothing too intense. Also, RAID-1 is mirrored, I think you're getting confused with RAID-0
Back to top
View user's profile Send private message
XQYZ
Apprentice
Apprentice


Joined: 19 Jul 2009
Posts: 231
Location: Europe

PostPosted: Sat May 08, 2010 6:15 am    Post subject: Reply with quote

frostschutz wrote:
I have another older 1TB green as external backup disk, which is slower...


Well, unless it's esata there's your reason.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Sat May 08, 2010 10:34 am    Post subject: Reply with quote

XQYZ wrote:
Well, unless it's esata there's your reason.


obviously it's esata...
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Sat May 08, 2010 10:57 am    Post subject: Reply with quote

Caligatio wrote:
I think you're getting confused with RAID-0


indeed, that's why i wasn't catching why doing that :)
Back to top
View user's profile Send private message
Akkara
Bodhisattva
Bodhisattva


Joined: 28 Mar 2006
Posts: 6702
Location: &akkara

PostPosted: Sat May 08, 2010 12:07 pm    Post subject: Reply with quote

I have two and they are working fine. Just be sure to use a recent version of parted - 2.2 or better. See bug 316453.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Sat May 08, 2010 1:02 pm    Post subject: Reply with quote

I used parted 2.2 but even so I had to align the partitions manually for this drive.

reading that bug report, it didn't "fight" me though, it just didn't do the alignment for me - maybe it's different for traditional DOS partitions, I used GPT this time around
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1759
Location: PB, Germany

PostPosted: Mon Apr 15, 2013 8:01 am    Post subject: Reply with quote

Hi. What is the state of today for Advanced Format harddisks?
I got a SAMSUNG SpinPoint M8:
Code:
Model Family:     SAMSUNG SpinPoint M8 (AF)
Device Model:     SAMSUNG HN-M101MBB
Serial Number:    S2RXJ9BB804573
LU WWN Device Id: 5 0024e9 205e897d9
Firmware Version: 2AR10001
User Capacity:    1.000.204.886.016 bytes [1,00 TB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 3.0, 3.0 Gb/s (current: 1.5 Gb/s)
Local Time is:    Fri Nov 23 15:11:42 2012 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

With stable smartmontools-5.42 there was no line like Family and pointing to (AF) but now with the latest smartmontools-6.0. The reported logical/physical sectores are 512 anyway and likely to be wrong.
I had problem with that 2,5" drive inside an old Dreambox 7020S (some 2.6.9 kernel) using some SATA-PATA converter. When I/O was increasing or even about after 30 minutes playback the kernel had I/O-errors and no response anymore. So idea was that alignment could be bad for the AF.

So how should these drives be partitioned today? I repartitioned and formatted (ext3) the drive from my Gentoo box. Old partition table showed from fdisk -lu /dev/sdb: Start=1 End=1953520064, so probably not aligned.
So I read these links and tried the common tools fdisk, cfdisk and parted:
  • https://www.tolaris.com/2011/07/21/libblkid-or-why-you-dont-need-to-worry-about-4k-disk-format/
    Actually every recent linux with recend libblkid should make it right.

  • http://linuxconfig.org/linux-wd-ears-advanced-format
    But using fdisk I can't choose starting sector 1, out of range. Same about 64. 2048 is possible but this would cost some unused space for bootloader I don't need on that drive. cfdisk starts from 63 or in maximizing mode 1. Both are not aligned to 8, so 'start MOD 8' is not 0. Finally I got it starting from 8 with fdisk expert menu, and moving "start of data" to 8.

  • http://johannes-bauer.com/linux/wdc/?menuid=3
    What about the end? Some guides about AF say this should also be divisible by 8, but there it is said that at least for fdisk it should give MOD 8 = 7 because of the inclusive notation of end sector in fdisk.
    Finally I have now this partition on fdisk:
    Start 8 end 1953525167
    ..while cfdisk shows this as:
    First Sector 0
    Last Sector 1953525167
    Offset 8
    Length 1953525168
Is this aligned or not?
No matter I had the same crashes again after installing the drive into Dreambox 7020S again. This could also be a damaged SATA-PATA adapter. but the drive is working fine connected to my Gentoo box via USB->SATA->S-/P-ATA-adaptor->drive, so integrating the adapter into chain when copying back the 100G back after re-partitioning.

After all is this AF thing also part of the kernel? Do correctly aligned partitions work with old kernels? Do not-aligned partitions have problems with every kernel, not matter if recent or old?
AFAIK the recent libblkid only helps when using fdisk to make it align correctly. But how should libblkid do so if the device does not report sector size correctly but 512? At least recent smartmontools do know about the AF familiy by some database...

EDIT: Then what about raw formatting without partition table at all? If I have no need for more then one partition I usually even don't use partition table at all. But then there is no way to do alignment. The filesystem (here ext3) is of course already advised to have 4096 block size, but if it is not aligned to the physical ones there is no way like moving the partition to start at 8.
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770
Back to top
View user's profile Send private message
srs5694
Guru
Guru


Joined: 08 Mar 2004
Posts: 434
Location: Woonsocket, RI

PostPosted: Mon Apr 15, 2013 4:56 pm    Post subject: Reply with quote

Massimo B. wrote:
Hi. What is the state of today for Advanced Format harddisks?


The issues are basically the same as they've been since Advanced Format disks first appeared. Software tends to use saner defaults today, though.

Quote:
I got a SAMSUNG SpinPoint M8:


I can't comment on that model specifically. Check with the manufacturer to learn if it's an Advanced Format model. If there are any doubts in your mind, though, you should assume that it is such a disk.

Quote:
I had problem with that 2,5" drive inside an old Dreambox 7020S (some 2.6.9 kernel) using some SATA-PATA converter. When I/O was increasing or even about after 30 minutes playback the kernel had I/O-errors and no response anymore. So idea was that alignment could be bad for the AF.


Misalignment on an Advanced Format drive is likely to result in consistently bad performance, not changes in performance over time. That said, I can't rule out the possibility that misalignment was at least part of your problem.

Quote:
So how should these drives be partitioned today?


Ensure that every partition begins on a sector that's a multiple of 8. That's it. Really.

Quote:
I repartitioned and formatted (ext3) the drive from my Gentoo box. Old partition table showed from fdisk -lu /dev/sdb: Start=1 End=1953520064, so probably not aligned.


If fdisk is showing a start sector of 1, then that's suspicious. Normally, the start sector of the first partition will be 63 or 2048. Other values are legal, but unusual. A start sector of 1 is most commonly associated with a GPT disk's protective MBR partition, which has a type of 0xEE. If that's what you're seeing, you should not use fdisk, since it doesn't support GPT. (Not yet, anyhow; apparently GPT support has been added to the development version, but AFAIK it's not yet been released.) Instead, use gdisk or parted on a GPT disk.

Quote:
So I read these links and tried the common tools fdisk, cfdisk and parted:[list][*]https://www.tolaris.com/2011/07/21/libblkid-or-why-you-dont-need-to-worry-about-4k-disk-format/
Actually every recent linux with recend libblkid should make it right.


I disagree with that article. libblkid is not a panacea. Programs can link to libblkid for any number of reasons, and are not obligated to use any partition-alignment advice that the library makes. What's more, the last I checked, libblkid relied on the hard disk to tell the truth about its characteristics, and many disks lie about that, so it's not really reliable. As proof that it's possible to mess things up even with a tool that uses libblkid, consider this:

Code:

$ ldd /sbin/fdisk
   linux-vdso.so.1 =>  (0x00007fff5cdff000)
   libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1 (0x00007fe4e0b08000)
   libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe4e0749000)
   libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fe4e0543000)
   /lib64/ld-linux-x86-64.so.2 (0x00007fe4e0d46000)


...so the fdisk on this system uses libblkid...

Code:

# fdisk /dev/sda

Command (m for help): u
Changing display/entry units to cylinders (DEPRECATED!)

Command (m for help): c
DOS Compatibility flag is set (DEPRECATED!)


This launched fdisk on an Advanced Format disk and sets it to use cylinder alignment with DOS compatibility mode.

Code:

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First cylinder (1-243201, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-243201, default 243201):
Using default value 243201

Command (m for help): u
Changing display/entry units to sectors

Command (m for help): p

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1              63  3907024064  1953512001   83  Linux


Now I've created a partition, switched back to sector display, and viewed the results: A partition that begins on sector 63, which is not properly aligned, despite the fact that fdisk uses libblkid.

Now, granted, in this example I had to jump through some hoops to get the program to do the wrong thing. It's not always that hard, though. For instance, fdisk only aligns its first partition automatically. If you start with a disk that already has one misaligned partition, subsequent partitions are likely to be misaligned, too.

[*]http://linuxconfig.org/linux-wd-ears-advanced-format
But using fdisk I can't choose starting sector 1, out of range. Same about 64. 2048 is possible but this would cost some unused space for bootloader I don't need on that drive. cfdisk starts from 63 or in maximizing mode 1. Both are not aligned to 8, so 'start MOD 8' is not 0. Finally I got it starting from 8 with fdisk expert menu, and moving "start of data" to 8.[/quote]

I don't see a date on the article to which you linked, and that's important, since the relevant Linux tools have changed since Advanced Format disks have arrived on the scene. Recent versions of fdisk start the first partition at sector 2048. Do not get hung up on the "lost" 1MiB of space! That amount of disk space is puny by today's standards. On your disk, for instance, it's 0.0001% of your available disk space! GRUB will also use at least some of that space, at least if the disk is bootable via the BIOS version of GRUB.

The last I heard, cfdisk was the last holdout for the old-style cylinder alignment, which is likely to create problems. Don't use it.

Quote:
What about the end? Some guides about AF say this should also be divisible by 8, but there it is said that at least for fdisk it should give MOD 8 = 7 because of the inclusive notation of end sector in fdisk.


Don't worry about partition end points. The speed problems of misalignment are due to filesystem data structures being placed inoptimally relative to the physical sector sizes, and filesystem data structures are laid out relative to the start points of partitions. The end points are relevant only in determining the filesystem size. The worst that will happen if you set partition end points "incorrectly" is that the filesystem will be slightly smaller than the partition and you'll have a small gap between partitions. To minimize both these factors, set the partition end point to one sector less than the optimal start point for the next partition. This is easily done in most tools by specifying a partition size that's a multiple of 8 sectors. Using notation like "+200G" in fdisk will do this, since that means 200GiB, which of course is a multiple of 8 sectors (4KiB).

Quote:
Finally I have now this partition on fdisk:
Start 8 end 1953525167
..while cfdisk shows this as:
First Sector 0
Last Sector 1953525167
Offset 8
Length 1953525168
Is this aligned or not?


The fdisk output suggests that it is, but the cfdisk output is less clear. I'd be inclined to trust fdisk rather than cfdisk. OTOH, starting the partition at sector 8 means that you probably won't be able to install BIOS-mode GRUB on the disk, which may be undesirable. I recommend changing the layout to start at sector 2048 instead, which is the default. This also has the advantage that you're not doing weird things with fdisk to override its defaults, which are sane, at least when you start a new partition table on the disk. Note that I (deliberately) created a non-optimal layout by overriding those defaults.

Quote:
After all is this AF thing also part of the kernel?


Not really. Alignment is a hardware issue within the hard disk, and is independent of kernels and OSes. You need to worry about it when using partitioning software -- a fact that disk manufacturers gloss over in favor of inaccurate generalities like "it's all OK if you use Windows Vista or later."

Quote:
Do correctly aligned partitions work with old kernels? Do ot-aligned partitions have problems with every kernel, not matter if recent or old?


The answer to both questions is "yes"; however, it's conceivable that a kernel might include features to minimize the problems associated with mis-aligned partitions. I don't know offhand if any Linux kernel includes such features, though.

Quote:
AFAIK the recent libblkid only helps when using fdisk to make it align correctly. But how should libblkid do so if the device does not report sector size correctly but 512? At least recent smartmontools do know about the AF familiy by some database...


If the disk lies, then there are only two ways libblkid or any software can deal with it: By incorporating a mammoth disk device database or by assuming that all disks are Advanced Format disks. As I said earlier, too, you should not assume that something using libblkid will handle things right in all cases. On the flip side, you shouldn't assume that something that does not use libblkid will do the wrong thing. My own gdisk program, for instance, doesn't use libblkid, but it aligns to 2048-sector multiples by default, and for all partitions. (There are exceptions in gdisk involving small disks with existing partition tables that use different alignment, though.) I looked into libblkid some time ago, and I just didn't see anything that was really beneficial to gdisk.

Quote:
EDIT: Then what about raw formatting without partition table at all? If I have no need for more then one partition I usually even don't use partition table at all. But then there is no way to do alignment. The filesystem (here ext3) is of course already advised to have 4096 block size, but if it is not aligned to the physical ones there is no way like moving the partition to start at 8.


If you put a filesystem directly on the disk device, then it is effectively the same as using a partition that begins at sector 0, which is properly aligned. That said, I don't recommend using a disk in that way; it's non-standard, and a badly-written disk utility might become confused and do something Really Bad. I can't cite a specific example of such problems right now, but the mere possibility is enough for me to advise against this use, since using a standard MBR or GPT partitioning scheme is easy, and aligning partitions for same is easy. If you can divide by 8 and check for a remainder, you can do it.

The bottom line is this: Ensure that every partition (with the exception of any extended partition you might create) begins on a sector value that's a multiple of 8. Ignore libblkid. Ignore the end sector value. Don't jump through a lot of hoops to override the partitioning software's defaults unless those default violate the one important rule (multiple-of-8 start points) -- and if a program does violate that rule, you should upgrade it or ditch it. Your quest for knowledge is admirable, but you're over-analyzing the problem. It's really not that hard.
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 3509

PostPosted: Tue Apr 16, 2013 1:10 am    Post subject: Reply with quote

frostschutz wrote:
The pitfall with the WD15EARS is that it does not report its nature to the OS, so it looks like a normal 512byte sector disk. hence the partitioning tools and mkfs do not automatically use 4k alignment - you have to take care of that manually. You have to verify that every layer you put on the disk (in my case GPT -> cryptsetup/LUKS -> LVM -> XFS/reiserfs/ext23) uses 4k sectors. Not just during install but every time you change something too. The kernel does not help you there at all yet (I even tried 2.6.34-rc6), I guess their improved 4k support means on a block driver level and probably also only for disks that report themselves as 4k disks too...


So next question... Do the WD Caviar Red drives properly report that they're Advanced Partition drives?
If they report properly, it sounds like the tools will "Just Work (TM)"?
Is one partitioning tool safer to use than others? I seem to get here a descending Advanced Partition reliability order of parted, fdisk, cfdisk.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
srs5694
Guru
Guru


Joined: 08 Mar 2004
Posts: 434
Location: Woonsocket, RI

PostPosted: Tue Apr 16, 2013 2:37 am    Post subject: Reply with quote

depontius wrote:
So next question... Do the WD Caviar Red drives properly report that they're Advanced Partition drives?
If they report properly, it sounds like the tools will "Just Work (TM)"?
Is one partitioning tool safer to use than others? I seem to get here a descending Advanced Partition reliability order of parted, fdisk, cfdisk.


Just do the math! Dividing a sector number by 8 with a pocket calculator (or desktop app equivalent) is not hard!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum