| View previous topic :: View next topic |
| Author |
Message |
eleanor
l33t
Joined: 01 Nov 2004
Posts: 666
|
 Posted: Sat Oct 27, 2012 9:08 am Post subject: LUKS + LVM No /dev/mapper/ entries |
 |
|
Hi, I've just booted my computer, which has the root partition encrypted with LUKS and it also uses LVM. The partition scheme is as follows:
- /dev/sda1: boot xfs partition
- /dev/sda2: ext3 partition currently not used
- /dev/sda3: LUKS encrypted LVM parittion that includes the following LV:
- root: xfs / partition
- home: xfs /home partition
- swap: swap partition
I'm using the following grub.conf:
| Code: |
default 1
timeout 5
title=LinuxMain
root (hd0,0)
kernel /boot/kernel-tuxonice-3.0.35 root=/dev/sda3 ikmap=slovene.bin
initrd /boot/initramfs.cpio.gz
|
The initramfs contains the following /init script:
| Code: |
#!/bin/busybox sh
# init to execute after switching to real root
init=/sbin/init
# Parse the arguments passed to the kernel option in grub.conf
parse_kernel_args() {
local x
CMDLINE=`cat /proc/cmdline`
for param in $CMDLINE; do
case "${param}" in
root=*)
root_device="`echo "${param}" | cut -d'=' -f2`"
;;
ikmap=*)
kmap="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
;;
iswap=*)
swap_device="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
;;
esac
done
}
main() {
# path to search for binaries
export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
umask 0077
# create needed directories (for mountpoints)
#for dir in proc sys dev newroot; do mkdir -p /$dir; done
# mount needed filesystems
/bin/busybox mount -t proc proc /proc
/bin/busybox mount -t sysfs sysfs /sys
#/bin/busybox mount -t tmpfs tmpfs /dev
# parse grub's kernel arguments
parse_kernel_args
# load keymap if it exists
if [ -n "$kmap" ]; then
loadkmap < "/etc/${kmap}"
else
die "Error: keymap /etc/${kmap} does not exist."
fi
# create /dev/sda encrypted partition
/bin/busybox echo /bin/mdev > /proc/sys/kernel/hotplug
/bin/busybox mdev -s
# LUKS: decrypt the encrypted partition
/sbin/cryptsetup -T 5 luksOpen "${root_device}" system
# LVM: enabled the LVM partitions
/sbin/lvm vgscan
/sbin/lvm vgchange -ay
# mount the root filesystem
#/bin/busybox mount /dev/mapper/system /newroot
/bin/busybox mount /dev/mapper/vg-root /newroot
if [ "$?" -ne 0 ]; then
/sbin/cryptsetup luksClose system 2>/dev/null || cryptsetup remove system
die "Error: mount root failed, dm-crypt mapping closed."
fi
# unmount unneeded filesystems
/bin/busybox umount -l /proc
/bin/busybox umount -l /sys
#/bin/busybox umount -l /dev
# switch to root of another filesystem and start the init process
exec switch_root /newroot "${init}"
}
main
|
The system boots fine, the only problem is that there are no /dev/mapper/ entries.
| Quote: |
# ls -l /dev/mapper/
crw------- 1 root root 10, 236 Oct 26 19:26 control
|
But the system is mounted fine:
| Quote: |
# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 60G 14G 47G 22% /
/dev/mapper/vg-root 60G 14G 47G 22% /
|
In the above df command we can see that the /dev/mapper/vg-root should exists and be visible, but it isn't. If I try to reenabled the /dev/mapper devices, I get the following error:
| Quote: |
# cryptsetup luksOpen /dev/sda3 test
Enter passphrase for /dev/sda3:
Cannot use device /dev/sda3 which is in use (already mapped or mounted).
|
The vgscan and vgchange commands don't help either:
| Quote: |
# vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2
# vgchange -ay
5 logical volume(s) in volume group "vg" now active
# ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 236 Oct 26 19:26 control
|
Does anybody have any idea what's going on? How can I make the already mounted /dev/sda3 LUKS+LVM logical volumes visible.
Thank you |
|
| Back to top |
|
 |
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21586
|
| Posted: Sat Oct 27, 2012 3:55 pm Post subject: Re: LUKS + LVM No /dev/mapper/ entries |
|
|
| eleanor wrote: | The initramfs contains the following /init script:
| Code: |
#!/bin/busybox sh
main() {
# path to search for binaries
export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
umask 0077
/bin/busybox mount -t proc proc /proc
/bin/busybox mount -t sysfs sysfs /sys
#/bin/busybox mount -t tmpfs tmpfs /dev
|
|
No devtmpfs?
| eleanor wrote: | | Code: |
/bin/busybox echo /bin/mdev > /proc/sys/kernel/hotplug
|
|
You can probably rely on echo being a built-in here.
Why do you create functions for these, when you do not need to call them more than once nor use the ability to return from a function? Also, where do you define die?
| eleanor wrote: | The system boots fine, the only problem is that there are no /dev/mapper/ entries.
|
What do you use to manage /dev? If you are using a static dev, this is expected. If you use devtmpfs or a userspace device manager, then /dev/mapper should be fully operational. |
|
| Back to top |
|
|
eleanor
l33t
Joined: 01 Nov 2004
Posts: 666
|
| Posted: Sat Oct 27, 2012 4:21 pm Post subject: |
|
|
Hi,
I've updated the script into the following:
| Code: |
#!/bin/busybox sh
# init to execute after switching to real root
init=/sbin/init
# Die function if something goes wrong
die() {
info "Dropping you into a minimal shell:"
exec /bin/sh
}
# Parse the arguments passed to the kernel option in grub.conf
parse_kernel_args() {
local x
CMDLINE=`cat /proc/cmdline`
for param in $CMDLINE; do
case "${param}" in
root=*)
root_device="`echo "${param}" | cut -d'=' -f2`"
;;
ikmap=*)
kmap="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
;;
iswap=*)
swap_device="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
;;
esac
done
}
#
# Main Function
#
# path to search for binaries
export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
umask 0077
# create needed directories (for mountpoints)
#for dir in proc sys dev newroot; do mkdir -p /$dir; done
# mount needed filesystems
/bin/busybox mount -t proc proc /proc
/bin/busybox mount -t sysfs sysfs /sys
/bin/busybox mount -t devtmpfs none /dev
# parse grub's kernel arguments
parse_kernel_args
# load keymap if it exists
if [ -n "$kmap" ]; then
loadkmap < "/etc/${kmap}"
else
die "Error: keymap /etc/${kmap} does not exist."
fi
# create /dev/sda encrypted partition
echo /bin/mdev > /proc/sys/kernel/hotplug
/bin/busybox mdev -s
# LUKS: decrypt the encrypted partition
/sbin/cryptsetup -T 5 luksOpen "${root_device}" system
# LVM: enabled the LVM partitions
/sbin/lvm vgscan
/sbin/lvm vgchange -ay
# mount the root filesystem
#/bin/busybox mount /dev/mapper/system /newroot
/bin/busybox mount /dev/mapper/vg-root /newroot
if [ "$?" -ne 0 ]; then
/sbin/cryptsetup luksClose system 2>/dev/null || cryptsetup remove system
die "Error: mount root failed, dm-crypt mapping closed."
fi
# unmount unneeded filesystems
/bin/busybox umount -l /proc
/bin/busybox umount -l /sys
/bin/busybox umount -l /dev
echo "Devices1: " >> /newroot/var/log/initrd
/bin/busybox ls -l /dev/mapper/ >> /newroot/var/log/initrd
# switch to root of another filesystem and start the init process
exec switch_root /newroot "${init}"
|
But the entries are still not shown after boot. You can also see that I've printed the "Device1" and /dev/mapper/ contents into the /var/log/, which contains the following;
| Quote: |
Devices1:
total 0
crw------- 1 0 0 10, 236 Oct 27 18:12 control
brw------- 1 0 0 253, 0 Oct 27 18:13 system
brw------- 1 0 0 253, 4 Oct 27 18:13 vg-boot
brw------- 1 0 0 253, 2 Oct 27 18:13 vg-home
brw------- 1 0 0 253, 3 Oct 27 18:13 vg-root
brw------- 1 0 0 253, 1 Oct 27 18:13 vg-swap
brw------- 1 0 0 253, 5 Oct 27 18:13 vg-vmware
|
This shows us that before the "exec switch_root" the /dev/mapper contains the right entries, and it's my main system's UDEV (which is what I use) that screws with things.
This is what's being started on the system's boot:
| Quote: |
# rc-update show
bootmisc | boot
dbus | default
devfs | sysinit
dmcrypt | boot
dmesg | sysinit
fsck | boot
hostname | boot
hwclock | boot
keymaps | boot
killprocs | shutdown
local | default
localmount | boot
modules | boot
mount-ro | shutdown
mtab | boot
net.lo | boot
netmount | default
procfs | boot
root | boot
savecache | shutdown
swap | boot
sysctl | boot
termencoding | boot
udev | sysinit
udev-postmount | default
urandom | boot
|
You can see that I used UDEV. And I guess it's UDEV's fault that the /dev/mapper/ entries from the initrd are gone.
Do you have any idea how to preserve the /dev/mapper/ entries from initrd and only add additional entries in there when the system boots?
Thank you for all your help |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Sat Oct 27, 2012 5:27 pm Post subject: |
|
|
eleanor ... I'm not sure exactly what the issue is but I think it might be resolved by 'mount --move' ... so before 'exec switch_root /newroot "${init}"' and prior to umount /dev, add something like the following:
| Code: | if mountpoint -q /dev/pts ; then umount /dev/pts; fi
echo '' > /proc/sys/kernel/hotplug
mount --move /dev /newroot/dev |
(culled from better-initramfs). Also (unrelated) ...
| eleanor wrote: | | Code: | | /sbin/cryptsetup -T 5 luksOpen "${root_device}" system |
|
cryptsetup makes a call to udevadm, and though its not fatal the following (again, culled from better-initramfs) will prevent it
| Code: | | echo -e "#!/bin/sh\nexit 0" > /sbin/udevadm && chmod 755 /sbin/udevadm |
HTH & best ... khay |
|
| Back to top |
|
|
eleanor
l33t
Joined: 01 Nov 2004
Posts: 666
|
| Posted: Sat Oct 27, 2012 6:50 pm Post subject: |
|
|
Hi, the mount --move didn't really do the trick.
I think when the switch_root switches to the new root, it overwrites the whole /dev when the udev is started, but why doesn't the udev detect LVM then?
Do you know if this can be related? |
|
| Back to top |
|
|
eleanor
l33t
Joined: 01 Nov 2004
Posts: 666
|
| Posted: Sat Oct 27, 2012 8:39 pm Post subject: |
|
|
Hi,
I guess I was right. If I start lvm I get the following:
| Quote: |
# /etc/init.d/lvm start
* Setting up the Logical Volume Manager ...
File descriptor 5 (/dev/ptmx) leaked on pvscan invocation. Parent PID 7339: /bin/sh
File descriptor 5 (/dev/ptmx) leaked on vgscan invocation. Parent PID 7339: /bin/sh
The link /dev/vg/swap should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/home should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/root should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/boot should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/vmware should had been created by udev but it was not found. Falling back to direct link creation.
File descriptor 5 (/dev/ptmx) leaked on vgchange invocation. Parent PID 7339: /bin/sh
|
This proves that the links should be added by udev once the switch_root is being called, but they aren't. The solution would be the following:
| Code: |
# rc-update add lvm boot
|
But I would still like to know why doesn't udev create the mappings in the first place. Any ideas? |
|
| Back to top |
|
|
truc
Advocate
Joined: 25 Jul 2005
Posts: 3199
|
| Posted: Fri Nov 23, 2012 12:30 am Post subject: |
|
|
Sorry for highjacking this thread, but I am having a similar problem with myroot being a btrfs subvolume of an LUKS encrypted device, /dev is managed with mdev inside the initramfs, the mapping is done correctly (e.g. I have /dev/mapper/luks_sdaX), but after switching root, I no longer have this mapping available which prevent me from mounting other subvolume!
From what I can read, this is because udev won't detect this mapping because it wasn't running yet when the mapping was done.
Does this mean I have to include udev in the initramfs? I mean, isn't there an other way given that udev is now included in systemd? What should I do?
Any advice greatly appreciated!
_________________
The End of the Internet! |
|
| Back to top |
|
|
truc
Advocate
Joined: 25 Jul 2005
Posts: 3199
|
| Posted: Fri Nov 23, 2012 1:44 pm Post subject: solved! |
|
|
I've found out that I can use dmsetup (from the lvm2 package) to create/remove mapping with the command
Here is the description:
| man dmsetup: | mknodes
[device_name]
Ensure that the node in /dev/mapper for device_name is cor‐
rect. If no device_name is supplied, ensure that all nodes
in /dev/mapper correspond to mapped devices currently loaded
by the device-mapper kernel driver, adding, changing or
removing nodes as necessary.
|
That's right, no need to fill /etc/mtab or whatever, just issue that command at boot and you're done(I've added a micro init service that does just that (inspired from the device-mapper one(again from the lvm2 package)):
| /etc/init.d/mydmsetup: | #!/sbin/runscript
depend() {
before checkfs fsck
after modules
before dmeventd
}
start() {
# ensure nodes are created
dmsetup mknodes
} |
Add it to theboot runlevel and that's it. I hope this will help others!
_________________
The End of the Internet! |
|
| Back to top |
|
|
cach0rr0
Bodhisattva
Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas
|
| Posted: Fri Nov 23, 2012 5:39 pm Post subject: |
|
|
FWIW - this isnt exclusive to lvm, btrfs subvolumes, or anything of the sort
i have a pretty boring vanilla setup
| Code: |
/dev/sda1 * 63 327682047 163840992+ 7 HPFS/NTFS/exFAT
/dev/sda2 * 327682048 327761919 39936 83 Linux
/dev/sda3 327761920 331970559 2104320 83 Linux
/dev/sda4 331970560 625141759 146585600 83 Linux
fdisk: cannot open /dev/mapper/root: No such file or directory
|
/dev/sda2 is boot (actually, i dont use that any more, as i have /boot on a thumb drive)
/dev/sda3 is swap (well, crypt-swap)
/dev/sda4 is root - i have no separate partitions, just everything on root (i had to resize my drives, TLDR, this was the only way to minimize downtime while doing a resize during a workday)
anyway, yall arent alone, and it isnt just your lvm/btrfs that's causing the issue.
All i have is:
| Code: |
hplaptop ~ # ls -alh /dev/mapper
total 0
drwxr-xr-x 2 root root 80 Nov 22 20:09 .
drwxr-xr-x 10 root root 5.9K Nov 22 20:09 ..
crw------- 1 root root 10, 236 Nov 22 20:09 control
brw------- 1 root root 253, 1 Nov 22 20:09 crypt-swap
|
aside from a few unsightly errors, it has made no *tangible* difference. But I wouldn't be able to fsck my stuff, so that could present an issue.
The one wrinkle for me - I don't use udev. At all. I use mdev,
| Code: |
alsasound |default
bootmisc | boot
consolefont | boot
dbus |default
devfs |sysinit
dmcrypt | boot default
dmesg |sysinit
fbcondecor |sysinit
fsck | boot
hostname | boot
hwclock | boot
keymaps | boot
killprocs |shutdown
local |default nonetwork
localmount | boot
mdev |sysinit
modules | boot
mount-ro |shutdown
mtab | boot
net.lo | boot
net.wlan0 |default
ntpd |default
procfs | boot
root | boot
savecache |shutdown
sshd |default
swap | boot
swapfiles | boot
sysctl | boot
syslog-ng |default
termencoding | boot
urandom | boot
vixie-cron |default
xdm |default
|
so mine may be a bit more self-inflicted. I think we're hitting a bug of some sort here, but I'm not sure quite what.
I may go the route of the "mini" init script above. It's just not pressing enough right now for me to really fiddle with it.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash |
|
| Back to top |
|
|
truc
Advocate
Joined: 25 Jul 2005
Posts: 3199
|
| Posted: Fri Nov 23, 2012 5:55 pm Post subject: |
|
|
you can try from the cli:
does output something for you? what about ?. if that's the case, just issue dmsetup mknodes and see how it goes.
_________________
The End of the Internet! |
|
| Back to top |
|
|
cach0rr0
Bodhisattva
Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas
|
| Posted: Fri Nov 23, 2012 9:25 pm Post subject: |
|
|
| truc wrote: | you can try from the cli:
does output something for you? what about ?. if that's the case, just issue dmsetup mknodes and see how it goes. |
aye, those are good to go
dmsetup mknodes does create the device nodes under /dev/mapper
since a)nothing is functionally wrong, and b)dmsetup mknodes is all i really need, i think ill probably go your route of just rolling my own init script to do the job
not too worried for me really, but somewhere in this must exist a bug worth filing - i just dont know *where* precisely, or rather, i dont know what component isnt doing its job.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Installing Gentoo
