Scripting parition creation

[grant123]

I'm scripting partition creation like this:

Code: Select all

fdisk /dev/sda <<EOF
d
1
d

n
p
1
2048
+64M

a
1

n
p
2
133120
250069679

w
EOF

Can this be made more dynamic? For example, how can I handle disks of various sizes and containing any number of partitions to be deleted?

[John R. Graham]

Kind of dangerous but you could delete the partition table with dd before you start. Should be okay since that's your intent anyway.

- John

[frostschutz]

in fdisk? o instead of d

parted can be used for scripting as well.

disk size is blockdev --getsize64 /dev/disk

[grant123]

Kind of dangerous but you could delete the partition table with dd before you start. Should be okay since that's your intent anyway.

Thanks, that should take care of deleting current paritions:

Code: Select all

dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=dd_image_file

Any idea on choosing default (like pressing Enter) instead of 133120 and 250069679?

[grant123]

in fdisk? o instead of d

I like that better. "DOS partition table" is synonymous with "partition table" then?

disk size is blockdev --getsize64 /dev/disk

That's strange. I get a very different number from this as compared to "End" reported in fdisk.

[maxim.251]

[/code]

Code: Select all

# Configure hard disk
echo "The following is a list of found Hard Disk devices (sda/hda):"
ls /dev/ | grep -E [s:h]da$
echo
echo "Please enter the device you would like to install to:"
read HD_DEVICE
HDD="/dev/${HD_DEVICE}"     #<--- hda or hdb or sda or sdc

# warn user last time
echo
echo "THIS IS YOUR LAST CHANCE TO CANCEL, Press CTRL+C to cancel this install or ENTER to continue"
read
echo "p" | fdisk $HDD       #<---- is wery dengerous, is deleting ewerything what you have on disk. do not use if you have system on de disk.                     #<----- denger
echo "Please specifiy the total number of partitions:"          #<---- denger
read PARTS                                                                     #<----- denger,

#TODO: add integer checking
echo -n "Generating fdisk operation file"
# Create fdisc auto file
((i = 1))
while (( i < PARTS ))
do
echo "d" >> fdisc.in
echo "$i" >> fdisc.in
((i += 1))
done
echo "d"    >> fdisc.in   # Delete last sector     
echo "n"    >> fdisc.in   # New Partiton
echo "p"    >> fdisc.in   # Primary
echo "1"    >> fdisc.in   # Partion 1
echo ""    >> fdisc.in   # default
echo "+32M"    >> fdisc.in   # 32 MB size
echo "a"    >> fdisc.in   # Set flag
echo "1"    >> fdisc.in   # bootable
echo -n "."
echo "n"    >> fdisc.in   # New Partion
echo "p"    >> fdisc.in   # Primary
echo "2"    >> fdisc.in   # Partion 2
echo ""    >> fdisc.in   # default
echo "+512M"    >> fdisc.in   # 512 MB size
echo "t"    >> fdisc.in   # Set partition type
echo "2"    >> fdisc.in   # Partition 2
echo "82"    >> fdisc.in   # 82 = SWAP
echo -n "."
echo "n"    >> fdisc.in   # New Partition
echo "p"    >> fdisc.in   # Primary
echo "3"    >> fdisc.in   # Partition 2
echo ""    >> fdisc.in   # default
echo ""    >> fdisc.in   # new Line
echo -n "."
echo "w"    >> fdisc.in   # Write partion table
echo "q"    >> fdisc.in   # Quit
echo ". Done"
# Execute file
echo "Executing fdisk script ..."
echo
fdisk $HDD < fdisc.in
#clean up
#rm -f fdisc.in  
echo ""
echo "Partions created"
echo "Applying filesystem to partitions"
mke2fs /dev/${HD_DEVICE}1
mke2fs -j /dev/${HD_DEVICE}3
mkswap /dev/${HD_DEVICE}2
echo "Activating swap partition"
swapon /dev/${HD_DEVICE}2

Be cerfull, if you do not know how is works this code, but it kan bee helpfull for you.

[maxim.251]

you have an error in your

Code: Select all

fdisk /dev/sda <<EOF 
d 
1 
d 

n 
p 
1 
2048 
+64M 

a 
1 

n 
p 
2 
133120 
250069679 

w 
EOF

fdisk /dev/sda <<-EOF


youre code


My egzampel is lookt like this

Code: Select all

passwd<<-PAS
1
1
PAS
echo "twoje hasło to 1"'

passwd maxim<<-PAS1
1
1
PAS1

[grant123]

Thank you, but where is the error in my code?

[fpemud]

I'm using python to do this kind of work in my project.
I think you can parse the output of "fdisk -l" for dynamic scenario.

Code: Select all

class FvmUtil:

	@staticmethod
	def shell(cmd, flags=""):
		"""Execute shell command"""

		assert cmd.startswith("/")

		# Execute shell command, throws exception when failed
		if flags == "":
			retcode = subprocess.Popen(cmd, shell = True).wait()
			if retcode != 0:
				raise Exception("Executing shell command \"%s\" failed, return code %d"%(cmd, retcode))
			return

		# Execute shell command, throws exception when failed, returns stdout+stderr
		if flags == "stdout":
			proc = subprocess.Popen(cmd,
									shell = True,
									stdout = subprocess.PIPE,
									stderr = subprocess.STDOUT)
			out = proc.communicate()[0]
			if proc.returncode != 0:
				raise Exception("Executing shell command \"%s\" failed, return code %d, output %s"%(cmd, proc.returncode, out))
			return out

		# Execute shell command, returns (returncode,stdout+stderr)
		if flags == "retcode+stdout":
			proc = subprocess.Popen(cmd,
									shell = True,
									stdout = subprocess.PIPE,
									stderr = subprocess.STDOUT)
			out = proc.communicate()[0]
			return (proc.returncode, out)

		assert False

	@staticmethod
	def shellInteractive(cmd, strInput, flags=""):
		"""Execute shell command with input interaction"""

		assert cmd.startswith("/")

		# Execute shell command, throws exception when failed
		if flags == "":
			proc = subprocess.Popen(cmd,
									shell = True,
									stdin = subprocess.PIPE)
			proc.communicate(strInput)
			if proc.returncode != 0:
				raise Exception("Executing shell command \"%s\" failed, return code %d"%(cmd, proc.returncode))
			return

		# Execute shell command, throws exception when failed, returns stdout+stderr
		if flags == "stdout":
			proc = subprocess.Popen(cmd,
									shell = True,
									stdin = subprocess.PIPE,
									stdout = subprocess.PIPE,
									stderr = subprocess.STDOUT)
			out = proc.communicate(strInput)[0]
			if proc.returncode != 0:
				raise Exception("Executing shell command \"%s\" failed, return code %d, output %s"%(cmd, proc.returncode, out))
			return out

		# Execute shell command, returns (returncode,stdout+stderr)
		if flags == "retcode+stdout":
			proc = subprocess.Popen(cmd,
									shell = True,
									stdin = subprocess.PIPE,
									stdout = subprocess.PIPE,
									stderr = subprocess.STDOUT)
			out = proc.communicate(strInput)[0]
			return (proc.returncode, out)

		assert False

Usage example:

Code: Select all

FvmUtil.shellInteractive('/sbin/fdisk "%s"'%(usbFile), "n\np\n\n\n\nt\n7\nw", "stdout")		# create one NTFS primary partition (system id: 7)

ret = FvmUtil.shell('/sbin/fdisk -l "%s"'%(usbFile), "stdout")
m = re.search("([0-9]+) +([0-9]+) +[^ ]+ +7", ret, re.M)				# Start(%d), End(%d), Blocks, Id(7)
if m is None:
	raise Exception("createWinUsbImg failed, fdisk failed")
startSector = int(m.group(1))
offset = int(m.group(1)) * 512
size = (int(m.group(2)) - int(m.group(1))) * 512

Caution: The code above is GPLv3 licensed.

[grant123]

Thanks fpemud.

in fdisk? o instead of d

That actually doesn't seem to work for me. fdisk "p" looks good but I can't boot sysresccd from a USB stick if I use "o" in fdisk instead of "d".

This works:

Code: Select all

dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=/dev/sd#

[user]

Again,
use parted which can take commands via command line and write a pedantic shell wrapper of you choose.
Don't fiddle with fdisk and EOF piping, this is fault-prone.

# parted --script /dev/<disk> mklabel gpt
# parted --align=min --script /dev/<disk> mkpart primary ext4 2048s 100%
# parted --script /dev/<disk> unit s print

[grant123]

Thanks, I should probably switch to parted.

[srs5694]

I tend to agree that parted is likely to be better than fdisk for this, although sfdisk might be worth considering for some uses. (I've always found it a bit strange, but it might suit your needs.)

I'd also like to bring up another point, related to this:

"DOS partition table" is synonymous with "partition table" then?

No. There are several different types of partition tables. The two most common are the Master Boot Record (MBR; aka MS-DOS, DOS, or BIOS) partition table and the GUID Partition Table (GPT). The MBR format has been in use for about three decades, and is a mass of kludges and workarounds. GPT is newer and not nearly as ugly. More importantly, MBR uses 32-bit pointers, which means that it tops out at 2^32 sectors -- 2TiB, given the common 512-byte sector size. Also, GPT is the preferred (and in some cases required) partition table type for computers that boot with the newer EFI/UEFI firmware. Thus, if you expect to use your script with disks that are over 2TiB or with EFI-based computers, you should either be using GPT rather than MBR in all cases or support both partition table types.

If GPT support is an issue, fdisk is a poor choice. The very latest versions of fdisk do support GPT, but only in a rudimentary way; and the version you've got installed might not even support GPT. GNU parted supports both MBR and GPT, so if your script needs to support both partition table types, parted is certainly the easiest way to go. If you decide to go GPT-only, you could instead use sgdisk (part of the gptfdisk package in Gentoo), which is designed for use by scripts.

[grant123]

"DOS partition table" is synonymous with "partition table" then?

No.

I found that out the hard way. This works to wipe out the partition table though:

Code: Select all

dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=/dev/sd#

There are several different types of partition tables. The two most common are the Master Boot Record (MBR; aka MS-DOS, DOS, or BIOS) partition table and the GUID Partition Table (GPT). The MBR format has been in use for about three decades, and is a mass of kludges and workarounds. GPT is newer and not nearly as ugly. More importantly, MBR uses 32-bit pointers, which means that it tops out at 2^32 sectors -- 2TiB, given the common 512-byte sector size. Also, GPT is the preferred (and in some cases required) partition table type for computers that boot with the newer EFI/UEFI firmware. Thus, if you expect to use your script with disks that are over 2TiB or with EFI-based computers, you should either be using GPT rather than MBR in all cases or support both partition table types.

If GPT support is an issue, fdisk is a poor choice. The very latest versions of fdisk do support GPT, but only in a rudimentary way; and the version you've got installed might not even support GPT. GNU parted supports both MBR and GPT, so if your script needs to support both partition table types, parted is certainly the easiest way to go. If you decide to go GPT-only, you could instead use sgdisk (part of the gptfdisk package in Gentoo), which is designed for use by scripts.

Thank you, I'm sure I'll be delving into that sooner or later.

[srs5694]

This works to wipe out the partition table though:

Code: Select all

dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=/dev/sd#

That will wipe out the MBR partition table (and a few bytes into the following sector). This will only damage a GUID Partition Table (GPT), not destroy it. Also, you've begun the wipe at the partition table itself, leaving the BIOS-mode boot loader intact; but the boot loader is probably useless without a partition table.

A better way to wipe any partition table is to use a tool designed for the job, like parted, fdisk, or gdisk. The details depend on your start point and intentions. Since most disks have partition tables, it's likely you want to replace an old partition table with a new one.

[grant123]

Is there a way to delete all existing partitions with fdisk regardless of how many there are?

I'm basically trying to get a totally fresh start and create a new set of partitions and make filesystems without anything hanging around from the previous install.

[cwr]

Is there a way to delete all existing partitions with fdisk regardless of how many there are?

I'm basically trying to get a totally fresh start and create a new set of partitions and make filesystems without anything hanging around from the previous install.

I generally write a couple of MB of zeros over the start of the drive to do this; however, this also wipes out the MBR boot loader code, so you need a replacement MBR (or a copy of your old one) to put in its place. How this works with a GPT I've no idea.

On the whole I'd just leave things alone and edit the partition table with one of the usual editors; though it's sometimes difficult to get good alignment with the current 4K/512B block size confusion.


Will

[Hu]

Aligning partitions to 1MB boundaries, though excessive and slightly wasteful, guarantees proper alignment regardless of whether your disk uses 512b or 4Kb sectors. As regards wiping the existing table and starting fresh, I think sfdisk and sgdisk do this by default when fed a partition table from stdin. Why are you trying to script an interactive editor when a non-interactive one is better at the job?

[cwr]

Aligning partitions to 1MB boundaries, though excessive and slightly wasteful, guarantees proper alignment regardless of whether your disk uses 512b or 4Kb sectors. As regards wiping the existing table and starting fresh, I think sfdisk and sgdisk do this by default when fed a partition table from stdin. Why are you trying to script an interactive editor when a non-interactive one is better at the job?

Yes, the problem is to force alignment. Most of the current partition editors seem to do a poor job of this.

Will

[Hu]

I think using sfdisk --force with a partition table on stdin does the right thing for MBR-style partitions even on unusual alignments.

[srs5694]

Is there a way to delete all existing partitions with fdisk regardless of how many there are?

Yes. The "o" option will do this, at least if it's started on an MBR disk. (If you launch fdisk on a GPT disk, it will adjust the protective MBR, but leave the bulk of the GPT data structures intact. This in turn will confuse parted.)

As others have said, though, fdisk is not the best tool for scripted use. Depending on your needs, I expect that parted, sfdisk, or maybe even sgdisk would be a better choice. See below for my specific suggestion.

I generally write a couple of MB of zeros over the start of the drive to do this; however, this also wipes out the MBR boot loader code, so you need a replacement MBR (or a copy of your old one) to put in its place. How this works with a GPT I've no idea.

I recommend breaking yourself of this habit. GPT stores its main partition table at the start of the disk and a backup at the end, so you're really only doing half the job by using dd to wipe the first MiB or two of the disk. GNU parted will interpret the result as an empty disk, but GPT fdisk won't; gdisk will ask if you want to recover the disk and sgdisk will refuse to touch it.

FWIW, a BIOS boot loader in the MBR is likely to be useless without a partition table, so the fact that your procedure wipes it out isn't a big deal.

Yes, the problem is to force alignment. Most of the current partition editors seem to do a poor job of this.

That's not really true. Current versions of libparted, GPT fdisk, and fdisk all align partitions to 2048-sector (1MiB) boundaries by default. IIRC, sfdisk requires that partitions be specified in sector units, so it's up to the user or the user's script to do this job, by design. If I'm not mistaken, cfdisk is the only major Linux tool that doesn't align partitions to a reasonable value by defaul, but should. Two or three years ago, your comment may have been accurate, but partitioning tools switched to 1MiB alignment in droves at around that time.

Getting back to the question of how to wipe a partition table, I recommend this:

Code: Select all

parted mklabel msdos /dev/sd{x}

Change "msdos" to "gpt" if GPT is preferred, and change "/dev/sd{x}" to the appropriate device.

This command will wipe the current partition table, no matter what it is, and create a new one of the specified type. This should eliminate old GPT data, which fdisk and sfdisk will not do, and you can easily change the command to create either GPT or MBR partition tables. You can add more parted commands to create new partitions, if desired. Overall, therefore, this is the single most flexible and fool-proof way to do the job. The fdisk and sfdisk tools are unreliable because they won't wipe the backup data. Ditto for dd, unless you bother to wipe the end of the disk.

In principle, you could use sgdisk to do it, as in "sgdisk -Z /dev/sd{x}"; however, if you plan to use MBR on the disk, you might as well use an MBR-capable program for the job rather than use sgdisk to wipe the partition table.

[cwr]

I tried using the current versions of fdisk and gparted a couple of months back, and got some odd results; the partitions seemed to be aligned, but there were unused gaps in the partition table entries and I couldn't find a way to build partitions of the right size to fill them. There probably is a way, but after an hour or two with several partition editors it wasn't obvious.

Will

[frostschutz]

in msdos logical partitions, it's normal to have 1MiB gaps for MiB-aligned partitions, since one sector is wasted for the partition information (and the alignment blows that up to a full MiB gap instead of 1-sector gap)

[beandog]

Aligning partitions to 1MB boundaries, though excessive and slightly wasteful, guarantees proper alignment regardless of whether your disk uses 512b or 4Kb sectors.

Where do you guys find these best practices for partitioning? I'm clueless, and genuinely curious.

[cwr]

in msdos logical partitions, it's normal to have 1MiB gaps for MiB-aligned partitions, since one sector is wasted for the partition information (and the alignment blows that up to a full MiB gap instead of 1-sector gap)

Thanks - that's probably the explanation for what I saw. It's just unfortunate that cfdisk is my favorite partition editor ...

Will