Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved] 4k sectors partitioning question
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
optiluca
Guru
Guru


Joined: 16 Jan 2006
Posts: 545
Location: Rivergaro, Italy

PostPosted: Tue Aug 03, 2010 10:18 am    Post subject: [solved] 4k sectors partitioning question Reply with quote

Hi all. I have recently purchased a laptop with a 4k sector, "advanced format" drive. I know these drives need to be treated carefully, namely ensuring that all partitions start on a sector multiple of 8 (even though I have seen other values such as 64 and 2048 floating around :? ). I am unsure however as to how to deal with logical partitions. Does only the "container" partition have to be aligned, or must they all start at sectors multiple of 8? I have also seen constrasting opinions regarding any measures that should be taken when formatting the partitions, with some claims that this operation needs to be "aligned" too. If so, how?

Thanks in advance, and apologies if this has already been asked before. I have seen a few 4k sector threads, but none really addressed these questions.

PS On a side note, should I use ext4 over ext3? Apart from a speeded up integrity check I really can't see anything of interest to a desktop user... :? Performance does not seem to be clearly better either, at least from what I have seen from Phoronix benchmarks... :roll:
_________________
# "Hmm, sounds like your system froze up."
# "I don't know why. It's about 80 degrees in here!"

http://www.rinkworks.com/stupid/cs_mincing.shtml


Last edited by optiluca on Wed Aug 04, 2010 6:56 am; edited 1 time in total
Back to top
View user's profile Send private message
richard.scott
Veteran
Veteran


Joined: 19 May 2003
Posts: 1497
Location: Oxfordshire, UK

PostPosted: Tue Aug 03, 2010 10:40 am    Post subject: Reply with quote

You could try this after emerging 'parted':

Code:
# parted --script /dev/sda mklabel gpt
# parted --align=min --script ${1} mkpart primary 0% 100%
# parted --script /dev/sda print


1. The mklabel gpt thing creates a GPT style partition table which seems to be required for bigger disks.
2. The --align option should align the partitions for you. Check the parted man page for other --align= options.

Rich.
Back to top
View user's profile Send private message
optiluca
Guru
Guru


Joined: 16 Jan 2006
Posts: 545
Location: Rivergaro, Italy

PostPosted: Tue Aug 03, 2010 11:22 am    Post subject: Reply with quote

The drive currently has a few windows partitions that I would like to keep, so I guess there's no touching the partition table. In any case, I have successfully used fdisk many times over and would be much more comfortable using fdisk & common sense vs. an "automagic" solution which could wreak havoc on my windows partitions. :? Could anyone confirm that using the "multiple of 8" strategy is indeed correct?

Thanks anyway :)
_________________
# "Hmm, sounds like your system froze up."
# "I don't know why. It's about 80 degrees in here!"

http://www.rinkworks.com/stupid/cs_mincing.shtml
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2971
Location: Germany

PostPosted: Tue Aug 03, 2010 12:17 pm    Post subject: Reply with quote

parted --align did not work for me (with an wd15ears drive that reports itself as being 512byte sector based).

In sectors, multiple of 8 works for 4k alignment, since one sector = 512 bytes, 8 sectors = 4k.

The container for the logical partitions does not matter, them partition themselves do. You have to align _all_ partitions that will later actually contain filesystems (or swap).

Or to explain the other way around:

- filesystems should use 4k blocksizes (careful when mkfs.)
- all storage layers below the filesystem (lvm, raid, encryption, partitions, ...) have to be 4k aligned so it ends up 4k aligned on the physical disk itself

I used parted (although manually). Mainly because I'm using GPT partitions, with traditional dos partitions I'd probably make the partition layout with cfdisk, and verify / fix the 4k alignment with sfdisk.
Back to top
View user's profile Send private message
optiluca
Guru
Guru


Joined: 16 Jan 2006
Posts: 545
Location: Rivergaro, Italy

PostPosted: Tue Aug 03, 2010 12:59 pm    Post subject: Reply with quote

Ok thank you very much :) One last doubt however. What blocksize does ext3 normally use? I would have guessed 512 bytes but according to http://linux.die.net/man/8/mke2fs , that is not even an option. How does the filesystem block size relate to the physical size? Is the idea that the filesystem block size must be at least the physical size?

Quote:
-b block-size
Specify the size of blocks in bytes. Valid block size vales are 1024, 2048 and 4096 bytes per block.


Thanks to all again :)
_________________
# "Hmm, sounds like your system froze up."
# "I don't know why. It's about 80 degrees in here!"

http://www.rinkworks.com/stupid/cs_mincing.shtml
Back to top
View user's profile Send private message
hielvc
Advocate
Advocate


Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca

PostPosted: Tue Aug 03, 2010 5:48 pm    Post subject: Reply with quote

-b block-size
Specify the size of blocks in bytes. Valid block size vales are 1024, 2048 and 4096 bytes per block. If omitted, mke2fs block-size is heuristically determined by the file system size and the expected usage of the filesystem (see the -T option). If block-size is negative, then mke2fs will use heuristics to determine the appropriate block size, with the constraint that the block size will be at least block-size bytes. This is useful for certain hardware devices which require that the blocksize be a multiple of 2k.


mke2fs -b 4096 ... shoud do it.
_________________
An A-Z Index of the Linux BASH command line
Back to top
View user's profile Send private message
optiluca
Guru
Guru


Joined: 16 Jan 2006
Posts: 545
Location: Rivergaro, Italy

PostPosted: Tue Aug 03, 2010 5:51 pm    Post subject: Reply with quote

I figured, I was merely puzzled as to what one does with a standard 512 byte sector considering that is not even an option...

BTW all my preinstalled windows partitions have a starting sector which is multiple of 2048 (with the first partition beginning on 2048 exactly). Why is that? :?
_________________
# "Hmm, sounds like your system froze up."
# "I don't know why. It's about 80 degrees in here!"

http://www.rinkworks.com/stupid/cs_mincing.shtml
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 18116

PostPosted: Wed Aug 04, 2010 2:45 am    Post subject: Reply with quote

optiluca wrote:
I figured, I was merely puzzled as to what one does with a standard 512 byte sector considering that is not even an option...
The hardware used 512 byte sectors for many years. There is no reason for the filesystem to use such small sectors. Smaller sectors require more metadata to track the same number of bytes, so there is a trade off between making sectors so small that the metadata wastes space on the drive or making sectors so large that the slack in the final sector of a file wastes an unacceptable amount of space.
optiluca wrote:
BTW all my preinstalled windows partitions have a starting sector which is multiple of 2048 (with the first partition beginning on 2048 exactly). Why is that? :?
That was probably done to ensure that they were aligned to the 4K boundary of the drive. As for why they are aligned to a larger multiple than strictly required, you would need to ask the vendor who prepared it that way.
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3117
Location: Canada

PostPosted: Wed Aug 04, 2010 3:11 am    Post subject: Reply with quote

Do align beginnings of all partitions on 2048 kbyte (2 MB) boundaries - both extended container, and logical partitions inside *). I find it easiest (I am old-fashioned) to do it with fdisk -u (which shows everything in sectors 512 sectors, you can switch to this view from inside fdisk as well).

*) It is not a 'must' - minimum you should try to align on 4k if it is mechanical drive (the first partition should start at sector 64 then)
and on 512k if it is SSD, but 2048k alignment is becoming standard, both Windows 7 and Mac OSX do this.


Correction: It is 1MB=1024k=2048 sectors alignment not 2MB, so align on 2048 sectors, not kilobytes.


Last edited by dmpogo on Wed Aug 04, 2010 6:05 am; edited 2 times in total
Back to top
View user's profile Send private message
optiluca
Guru
Guru


Joined: 16 Jan 2006
Posts: 545
Location: Rivergaro, Italy

PostPosted: Wed Aug 04, 2010 5:48 am    Post subject: Reply with quote

dmpogo wrote:
Do align beginnings of all partitions on 2048 kbyte (2 MB) boundaries - both extended container, and logical partitions inside *). I find it easiest (I am old-fashioned) to do it with fdisk -u (which shows everything in sectors 512 sectors, you can switch to this view from inside fdisk as well).


So to align to 2048 kbyte boundaries, that would require multiples of 4096? That is not what my current windows partitions are using :? Windows seems to be aligning to 1MB boundaries as it stands. I have now partitioned the drive following this same convention, beginning every single partition on a multiple of 2048. Hope that's right :D

Thanks to all again for your very prompt and exhaustive replies :)
_________________
# "Hmm, sounds like your system froze up."
# "I don't know why. It's about 80 degrees in here!"

http://www.rinkworks.com/stupid/cs_mincing.shtml
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 3117
Location: Canada

PostPosted: Wed Aug 04, 2010 5:57 am    Post subject: Reply with quote

optiluca wrote:
dmpogo wrote:
Do align beginnings of all partitions on 2048 kbyte (2 MB) boundaries - both extended container, and logical partitions inside *). I find it easiest (I am old-fashioned) to do it with fdisk -u (which shows everything in sectors 512 sectors, you can switch to this view from inside fdisk as well).


So to align to 2048 kbyte boundaries, that would require multiples of 4096? That is not what my current windows partitions are using :? Windows seems to be aligning to 1MB boundaries as it stands. I have now partitioned the drive following this same convention, beginning every single partition on a multiple of 2048. Hope that's right :D

Thanks to all again for your very prompt and exhaustive replies :)


I have misspoke and it is of course 1MB. Double checked, yes, my drive partitions, they do start at block 2048.
I guess I picked up on your 2048 in the opening post and mentally converted it into 2 MB :)

well, we through megabytes around, and I still have my first laptop operation which has 20 MB harddrive total. And it had windows, word, a book my wife wrote, all in that space.
Back to top
View user's profile Send private message
optiluca
Guru
Guru


Joined: 16 Jan 2006
Posts: 545
Location: Rivergaro, Italy

PostPosted: Wed Aug 04, 2010 6:58 am    Post subject: Reply with quote

Ok thanks for confirming that :) In that case I shall proceed with my installation, which has now encountered an immense circular dependency issue, but that is for another thread... :P

Thanks to all :D
_________________
# "Hmm, sounds like your system froze up."
# "I don't know why. It's about 80 degrees in here!"

http://www.rinkworks.com/stupid/cs_mincing.shtml
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Dec 10, 2010 1:16 pm    Post subject: Reply with quote

this should answer some questions:
Linux on 4KB-sector disks: Practical advice

with the latest parted (2.2 or 2.3) and gparted (0.7, probably also 0.6.2) you just need to follow the advice in the posted link or

leave it set at "Align to:" MiB (in gparted 0.7) - so that gparted will take care of that for you :)


sample result:
Quote:
Device Boot Start End Blocks Id System
/dev/sda1 2048 2050047 1024000 83 Linux
/dev/sda2 2050048 104450047 51200000 7 HPFS/NTFS

_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
Atha
Apprentice
Apprentice


Joined: 22 Sep 2004
Posts: 213

PostPosted: Wed Apr 13, 2011 3:36 pm    Post subject: Reply with quote

Hi!

I just faced the same problems with my new Hitachi Deskstar 5K3000 2 TB SATA-III (6Gb/s) 5400rpm hard drive. I already had a 2 TB Western Digital Caviar Green WD20EADS (512b blocks) which didn't need any adjustment/alignment – obviously. So with the Hitachi it is the first time I'm confronted with the new “Advanced Format Technology”.

Since the Hitachi Deskstar reports 512b blocks but realy has 4k blocks, the multiple of 8 rule seemed to be the only way to go.

Code:
# parted /dev/sdc
GNU Parted 2.3
Verwende /dev/sdc
Willkommen zu GNU Parted! Geben Sie 'help' ein, um eine Liste der verfügbaren Kommados zu erhalten.
(parted) unit s
(parted) print
Modell: ATA Hitachi HDS5C302 (scsi)
Festplatte  /dev/sdc:  3907029168s
Sektorgröße (logisch/physisch): 512B/512B
Partitionstabelle: gpt

Nummer  Anfang  Ende         Größe        Dateisystem  Name     Flags
 1      40s     3907029128s  3907029089s  ext4         primary

(parted) quit


(Sorry, German language)
I set the end also to be a multiple of 8. Don't know if this is really necessary, but it seemed logical to me.

The filesystem is Ext4, and I used mke2fs -T ext4 -t ext4 -b 4096 to create it.

How do I test the drives' performance?
Back to top
View user's profile Send private message
Delicates
n00b
n00b


Joined: 19 Jul 2005
Posts: 6

PostPosted: Sun Apr 24, 2011 12:09 pm    Post subject: Reply with quote

Atha wrote:

Code:

Nummer  Anfang  Ende         Größe        Dateisystem  Name     Flags
 1      40s     3907029128s  3907029089s  ext4         primary

I set the end also to be a multiple of 8. Don't know if this is really necessary, but it seemed logical to me.

The filesystem is Ext4, and I used mke2fs -T ext4 -t ext4 -b 4096 to create it.


The end sector number should be a multiple of 8 less by 1 (LBA numbering starts at 0), otherwise the size of your entire partition (3907029089s reported above) is not divisible by 8 sectors.

I'm currently slowly going through installation of the Advanced Format Technology (AFT) 3TB Hitachi Deskstar 7K3000 (HDS723030ALA640) under Gentoo Linux with a single ext4 partition.
The drive uses Advanced Format 512 emulation (512e) - that is even though the drive uses 4096 byte sector sizes, it reports 512 byte sectors to OS/BIOS/Apps, not 4k.

Code:

# dmesg | egrep 'ata5|sd 4|sdd'
ata5: SATA max UDMA/133 abar m2048@0xfbffc000 port 0xfbffc300 irq 43
ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata5.00: ATA-8: Hitachi HDS723030ALA640, MKAOA3B0, max UDMA/133
ata5.00: 5860533168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata5.00: configured for UDMA/133
sd 4:0:0:0: [sdd] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)
sd 4:0:0:0: Attached scsi generic sg4 type 0
sd 4:0:0:0: [sdd] Write Protect is off
sd 4:0:0:0: [sdd] Mode Sense: 00 3a 00 00
sd 4:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdd: unknown partition table
sd 4:0:0:0: [sdd] Attached SCSI disk


Of course initially I had to ensure required Kernel ( 2.6.38 ) configuration is enabled:

Code:

Enable the block layer (BLOCK)  --->
     [*] Support for large (2TB+) block devices and files (LBDAF)
File systems  --->
     [*] The Extended 4 (ext4) filesystem
     Partition Types  --->
          [*] Advanced Partition Selection (PARTITION_ADVANCED)  --->
               [*] EFI GUID Partition Support (EFI_PARTITION)


LBDAF option is required for 32bit systems only, and is not needed/available for 64bit.

Then I tried using Gnome Disk Utility, but while it allowed creation of a GUID Partition Table (GPT) it did not provide any control over partition alignment.
It was good however for performing an initial thorough scan of disk surface for errors and the SMART report.

I found GPT fdisk (gdisk) to be slightly easier to use than GNU Parted (parted), as it allows you to set a partition alignment value (default is 2048 sectors or 1MiB, which I changed to 8 sectors), the alignment value works on partition start, but not on partition end, so I had to specify the right last sector manually:

Code:

# gdisk /dev/sdd
GPT fdisk (gdisk) version 0.7.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): x

Expert command (? for help): d
Partitions will begin on 2048-sector boundaries.

Expert command (? for help): l 8
Enter the sector alignment value (1-65536, default = 2048): 8

Expert command (? for help): d
Partitions will begin on 8-sector boundaries.

Expert command (? for help): m

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-5860533134, default = 34) or {+-}size{KMGTP}:
Information: Moved requested sector from 34 to 40 in
order to align on 8-sector boundaries.
Use 'l' on the experts' menu to adjust alignment
Last sector (40-5860533134, default = 5860533134) or {+-}size{KMGTP}: 5860533127
Current type is 'Linux/Windows data'
Hex code or GUID (L to show codes, Enter = 0700):
Changed type of partition to 'Linux/Windows data'

Command (? for help): i
Using 1
Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Linux/Windows data)
Partition unique GUID: 8D8AF9DF-9C0B-4787-8003-86986EF1EC5F
First sector: 40 (at 20.0 KiB)
Last sector: 5860533127 (at 2.7 TiB)
Partition size: 5860533088 sectors (2.7 TiB)
Attribute flags: 0000000000000000
Partition name: 'Linux/Windows data'

Command (? for help): c
Using 1
Enter name: STORAGE

Command (? for help): p
Disk /dev/sdd: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): C49984B5-9824-42EA-9F0E-B6BEC4A41F38
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 8-sector boundaries
Total free space is 13 sectors (6.5 KiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              40      5860533127   2.7 TiB     0700  STORAGE

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT).
The operation has completed successfully.

# parted /dev/sdd
GNU Parted 2.3
Using /dev/sdd
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) u s                                                             
(parted) p                                                               
Model: ATA Hitachi HDS72303 (scsi)
Disk /dev/sdd: 5860533168s
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start  End          Size         File system  Name     Flags
 1      40s    5860533127s  5860533088s               STORAGE

(parted) q                                                               

#


And finally formatting with ext4.
As this drive is purely for storage and is not for any system partition - don't reserve anything for super user, allowing normal users to fill the entire drive, and disabling huge_file support/overhead as I doubt I'll store files larger than 2TB on this 3TB drive. Also even on smaller drives, 32bit systems would not be able to mount the default huge_file ext4 file systems without LBDAF kernel setting enabled. If needed huge_file can be enabled later with tune2fs.

Code:

# mkfs.ext4 -m 0 -O ^huge_file -L STORAGE /dev/sdd1

mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=STORAGE
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
183148544 inodes, 732566636 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
22357 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000, 550731776, 644972544

Writing inode tables: done                           
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 29 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

# tune2fs -l /dev/sdd1

tune2fs 1.41.14 (22-Dec-2010)
Filesystem volume name:   STORAGE
Last mounted on:          <not available>
Filesystem UUID:          5a2179ea-884a-44fd-9af9-43dfe9caa41c
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              183148544
Block count:              732566636
Reserved block count:     0
Free blocks:              721019814
Free inodes:              183148533
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      849
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Sun Apr 24 21:23:26 2011
Last mount time:          n/a
Last write time:          Sun Apr 24 21:23:36 2011
Mount count:              0
Maximum mount count:      29
Last checked:             Sun Apr 24 21:23:26 2011
Check interval:           15552000 (6 months)
Next check after:         Fri Oct 21 22:23:26 2011
Lifetime writes:          152 MB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      bdcfb23e-5918-4742-94e5-aae3e5a0b23a
Journal backup:           inode blocks
#


After copying all files (using cp -av) from a filled up 1.5TB ext3 drive onto this new 3TB drive, I had 1.5GB of space saving reported by du on the new file system.
Interestingly diff -u of ls -lR output on both drives has shown that on ext4 the size of some directories (with 200+ files) increased (by one 4k block), while size of other (with 1500+ files) decreased (by 29 blocks).
Back to top
View user's profile Send private message
xdarma
l33t
l33t


Joined: 08 Dec 2003
Posts: 710
Location: tra veneto e friuli (italy)

PostPosted: Sun Oct 02, 2011 7:19 am    Post subject: Reply with quote

kernelOfTruth wrote:
this should answer some questions:
Linux on 4KB-sector disks: Practical advice

Thanks for the link (and thank you google for cache ;-), this is the fdisk command line:
Quote:
Another approach with fdisk is to launch it as fdisk -H 224 -S 56 /dev/sda, which changes the cylinder/head/sector (CHS) geometry to guarantee proper 4096-byte alignment when the program aligns to cylinders, as it does by default.

_________________
proud user of faKeDE-4.7.3 -> back to windowmaker -> moved to LXQt
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum