Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
LUKS + LVM No /dev/mapper/ entries
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
eleanor
l33t
l33t


Joined: 01 Nov 2004
Posts: 661

PostPosted: Sat Oct 27, 2012 9:08 am    Post subject: LUKS + LVM No /dev/mapper/ entries Reply with quote

Hi, I've just booted my computer, which has the root partition encrypted with LUKS and it also uses LVM. The partition scheme is as follows:
- /dev/sda1: boot xfs partition
- /dev/sda2: ext3 partition currently not used
- /dev/sda3: LUKS encrypted LVM parittion that includes the following LV:
- root: xfs / partition
- home: xfs /home partition
- swap: swap partition

I'm using the following grub.conf:

Code:

default 1
timeout 5

title=LinuxMain
root (hd0,0)
kernel /boot/kernel-tuxonice-3.0.35 root=/dev/sda3 ikmap=slovene.bin
initrd /boot/initramfs.cpio.gz


The initramfs contains the following /init script:

Code:

#!/bin/busybox sh

# init to execute after switching to real root
init=/sbin/init



# Parse the arguments passed to the kernel option in grub.conf
parse_kernel_args() {
  local x
  CMDLINE=`cat /proc/cmdline`
  for param in $CMDLINE; do
    case "${param}" in
      root=*)
        root_device="`echo "${param}" | cut -d'=' -f2`"
        ;;
      ikmap=*)
        kmap="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
        ;;
      iswap=*)
        swap_device="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
        ;;
      esac
    done
}


main() {
  # path to search for binaries
  export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
  umask 0077

  # create needed directories (for mountpoints)
  #for dir in proc sys dev newroot; do mkdir -p /$dir; done

  # mount needed filesystems
  /bin/busybox mount -t proc proc /proc
  /bin/busybox mount -t sysfs sysfs /sys
  #/bin/busybox mount -t tmpfs tmpfs /dev

  # parse grub's kernel arguments
  parse_kernel_args

  # load keymap if it exists
  if [ -n "$kmap" ]; then
    loadkmap < "/etc/${kmap}"
  else
    die "Error: keymap /etc/${kmap} does not exist."
  fi

  # create /dev/sda encrypted partition
  /bin/busybox echo /bin/mdev > /proc/sys/kernel/hotplug
  /bin/busybox mdev -s

  # LUKS: decrypt the encrypted partition
  /sbin/cryptsetup -T 5 luksOpen "${root_device}" system

  # LVM: enabled the LVM partitions
  /sbin/lvm vgscan
  /sbin/lvm vgchange -ay


  # mount the root filesystem
  #/bin/busybox mount /dev/mapper/system /newroot
  /bin/busybox mount /dev/mapper/vg-root /newroot
  if [ "$?" -ne 0 ]; then
    /sbin/cryptsetup luksClose system 2>/dev/null || cryptsetup remove system
    die "Error: mount root failed, dm-crypt mapping closed."
  fi

  # unmount unneeded filesystems
  /bin/busybox umount -l /proc
  /bin/busybox umount -l /sys
  #/bin/busybox umount -l /dev

  # switch to root of another filesystem and start the init process
  exec switch_root /newroot "${init}"
}
main



The system boots fine, the only problem is that there are no /dev/mapper/ entries.
Quote:

# ls -l /dev/mapper/
crw------- 1 root root 10, 236 Oct 26 19:26 control


But the system is mounted fine:
Quote:

# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 60G 14G 47G 22% /
/dev/mapper/vg-root 60G 14G 47G 22% /


In the above df command we can see that the /dev/mapper/vg-root should exists and be visible, but it isn't. If I try to reenabled the /dev/mapper devices, I get the following error:

Quote:

# cryptsetup luksOpen /dev/sda3 test
Enter passphrase for /dev/sda3:
Cannot use device /dev/sda3 which is in use (already mapped or mounted).


The vgscan and vgchange commands don't help either:

Quote:

# vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2

# vgchange -ay
5 logical volume(s) in volume group "vg" now active

# ls -l /dev/mapper/
total 0
crw------- 1 root root 10, 236 Oct 26 19:26 control




Does anybody have any idea what's going on? How can I make the already mounted /dev/sda3 LUKS+LVM logical volumes visible.

Thank you
Back to top
View user's profile Send private message
Hu
Watchman
Watchman


Joined: 06 Mar 2007
Posts: 9009

PostPosted: Sat Oct 27, 2012 3:55 pm    Post subject: Re: LUKS + LVM No /dev/mapper/ entries Reply with quote

eleanor wrote:
The initramfs contains the following /init script:
Code:

#!/bin/busybox sh

main() {
  # path to search for binaries
  export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
  umask 0077
  /bin/busybox mount -t proc proc /proc
  /bin/busybox mount -t sysfs sysfs /sys
  #/bin/busybox mount -t tmpfs tmpfs /dev
No devtmpfs?
eleanor wrote:
Code:

  /bin/busybox echo /bin/mdev > /proc/sys/kernel/hotplug
You can probably rely on echo being a built-in here.
eleanor wrote:
Code:

main
Why do you create functions for these, when you do not need to call them more than once nor use the ability to return from a function? Also, where do you define die?
eleanor wrote:
The system boots fine, the only problem is that there are no /dev/mapper/ entries.
What do you use to manage /dev? If you are using a static dev, this is expected. If you use devtmpfs or a userspace device manager, then /dev/mapper should be fully operational.
Back to top
View user's profile Send private message
eleanor
l33t
l33t


Joined: 01 Nov 2004
Posts: 661

PostPosted: Sat Oct 27, 2012 4:21 pm    Post subject: Reply with quote

Hi,

I've updated the script into the following:

Code:

#!/bin/busybox sh

# init to execute after switching to real root
init=/sbin/init


# Die function if something goes wrong
die() {
   info "Dropping you into a minimal shell:"
   exec /bin/sh
}


# Parse the arguments passed to the kernel option in grub.conf
parse_kernel_args() {
  local x
  CMDLINE=`cat /proc/cmdline`
  for param in $CMDLINE; do
    case "${param}" in
      root=*)
        root_device="`echo "${param}" | cut -d'=' -f2`"
        ;;
      ikmap=*)
        kmap="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
        ;;
      iswap=*)
        swap_device="`echo "${param}" | cut -d'=' -f2 | cut -d':' -f1`"
        ;;
      esac
    done
}



#
# Main Function
#
# path to search for binaries
export PATH="/sbin:/bin:/usr/bin:/usr/sbin"
umask 0077

# create needed directories (for mountpoints)
#for dir in proc sys dev newroot; do mkdir -p /$dir; done

# mount needed filesystems
/bin/busybox mount -t proc proc /proc
/bin/busybox mount -t sysfs sysfs /sys
/bin/busybox mount -t devtmpfs none /dev

# parse grub's kernel arguments
parse_kernel_args

# load keymap if it exists
if [ -n "$kmap" ]; then
  loadkmap < "/etc/${kmap}"
else
  die "Error: keymap /etc/${kmap} does not exist."
fi

# create /dev/sda encrypted partition
echo /bin/mdev > /proc/sys/kernel/hotplug
/bin/busybox mdev -s

# LUKS: decrypt the encrypted partition
/sbin/cryptsetup -T 5 luksOpen "${root_device}" system

# LVM: enabled the LVM partitions
/sbin/lvm vgscan
/sbin/lvm vgchange -ay


# mount the root filesystem
#/bin/busybox mount /dev/mapper/system /newroot
/bin/busybox mount /dev/mapper/vg-root /newroot
if [ "$?" -ne 0 ]; then
  /sbin/cryptsetup luksClose system 2>/dev/null || cryptsetup remove system
  die "Error: mount root failed, dm-crypt mapping closed."
fi

# unmount unneeded filesystems
/bin/busybox umount -l /proc
/bin/busybox umount -l /sys
/bin/busybox umount -l /dev


echo "Devices1: " >> /newroot/var/log/initrd
/bin/busybox ls -l /dev/mapper/ >> /newroot/var/log/initrd

# switch to root of another filesystem and start the init process
exec switch_root /newroot "${init}"




But the entries are still not shown after boot. You can also see that I've printed the "Device1" and /dev/mapper/ contents into the /var/log/, which contains the following;

Quote:

Devices1:
total 0
crw------- 1 0 0 10, 236 Oct 27 18:12 control
brw------- 1 0 0 253, 0 Oct 27 18:13 system
brw------- 1 0 0 253, 4 Oct 27 18:13 vg-boot
brw------- 1 0 0 253, 2 Oct 27 18:13 vg-home
brw------- 1 0 0 253, 3 Oct 27 18:13 vg-root
brw------- 1 0 0 253, 1 Oct 27 18:13 vg-swap
brw------- 1 0 0 253, 5 Oct 27 18:13 vg-vmware


This shows us that before the "exec switch_root" the /dev/mapper contains the right entries, and it's my main system's UDEV (which is what I use) that screws with things.

This is what's being started on the system's boot:
Quote:

# rc-update show
bootmisc | boot
dbus | default
devfs | sysinit
dmcrypt | boot
dmesg | sysinit
fsck | boot
hostname | boot
hwclock | boot
keymaps | boot
killprocs | shutdown
local | default
localmount | boot
modules | boot
mount-ro | shutdown
mtab | boot
net.lo | boot
netmount | default
procfs | boot
root | boot
savecache | shutdown
swap | boot
sysctl | boot
termencoding | boot
udev | sysinit
udev-postmount | default
urandom | boot


You can see that I used UDEV. And I guess it's UDEV's fault that the /dev/mapper/ entries from the initrd are gone.

Do you have any idea how to preserve the /dev/mapper/ entries from initrd and only add additional entries in there when the system boots?

Thank you for all your help
Back to top
View user's profile Send private message
khayyam
Advocate
Advocate


Joined: 07 Jun 2012
Posts: 2244

PostPosted: Sat Oct 27, 2012 5:27 pm    Post subject: Reply with quote

eleanor ... I'm not sure exactly what the issue is but I think it might be resolved by 'mount --move' ... so before 'exec switch_root /newroot "${init}"' and prior to umount /dev, add something like the following:

Code:
    if mountpoint -q /dev/pts ; then umount /dev/pts; fi
    echo '' > /proc/sys/kernel/hotplug
    mount --move /dev /newroot/dev

(culled from better-initramfs). Also (unrelated) ...

eleanor wrote:
Code:
/sbin/cryptsetup -T 5 luksOpen "${root_device}" system

cryptsetup makes a call to udevadm, and though its not fatal the following (again, culled from better-initramfs) will prevent it

Code:
echo -e "#!/bin/sh\nexit 0" > /sbin/udevadm && chmod 755 /sbin/udevadm

HTH & best ... khay
Back to top
View user's profile Send private message
eleanor
l33t
l33t


Joined: 01 Nov 2004
Posts: 661

PostPosted: Sat Oct 27, 2012 6:50 pm    Post subject: Reply with quote

Hi, the mount --move didn't really do the trick.

I think when the switch_root switches to the new root, it overwrites the whole /dev when the udev is started, but why doesn't the udev detect LVM then?

Do you know if this can be related?
Back to top
View user's profile Send private message
eleanor
l33t
l33t


Joined: 01 Nov 2004
Posts: 661

PostPosted: Sat Oct 27, 2012 8:39 pm    Post subject: Reply with quote

Hi,

I guess I was right. If I start lvm I get the following:

Quote:

# /etc/init.d/lvm start
* Setting up the Logical Volume Manager ...
File descriptor 5 (/dev/ptmx) leaked on pvscan invocation. Parent PID 7339: /bin/sh
File descriptor 5 (/dev/ptmx) leaked on vgscan invocation. Parent PID 7339: /bin/sh
The link /dev/vg/swap should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/home should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/root should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/boot should had been created by udev but it was not found. Falling back to direct link creation.
The link /dev/vg/vmware should had been created by udev but it was not found. Falling back to direct link creation.
File descriptor 5 (/dev/ptmx) leaked on vgchange invocation. Parent PID 7339: /bin/sh


This proves that the links should be added by udev once the switch_root is being called, but they aren't. The solution would be the following:

Code:

# rc-update add lvm boot


But I would still like to know why doesn't udev create the mappings in the first place. Any ideas?
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Fri Nov 23, 2012 12:30 am    Post subject: Reply with quote

Sorry for highjacking this thread, but I am having a similar problem with myroot being a btrfs subvolume of an LUKS encrypted device, /dev is managed with mdev inside the initramfs, the mapping is done correctly (e.g. I have /dev/mapper/luks_sdaX), but after switching root, I no longer have this mapping available which prevent me from mounting other subvolume!

From what I can read, this is because udev won't detect this mapping because it wasn't running yet when the mapping was done.

Does this mean I have to include udev in the initramfs? I mean, isn't there an other way given that udev is now included in systemd? What should I do?

Any advice greatly appreciated!
_________________
The End of the Internet!
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Fri Nov 23, 2012 1:44 pm    Post subject: solved! Reply with quote

I've found out that I can use dmsetup (from the lvm2 package) to create/remove mapping with the command
Code:
dmsetup mknodes


Here is the description:
man dmsetup:
mknodes
       [device_name]
       Ensure that the node in /dev/mapper for device_name  is  cor‐
       rect.   If  no device_name is supplied, ensure that all nodes
       in /dev/mapper correspond to mapped devices currently  loaded
       by  the  device-mapper  kernel  driver,  adding,  changing or
       removing nodes as necessary.


That's right, no need to fill /etc/mtab or whatever, just issue that command at boot and you're done(I've added a micro init service that does just that (inspired from the device-mapper one(again from the lvm2 package)):
/etc/init.d/mydmsetup:
#!/sbin/runscript

depend() {
   before checkfs fsck
   after modules
   before dmeventd
}

start() {
   # ensure nodes are created
   dmsetup mknodes
}


Add it to theboot runlevel and that's it. I hope this will help others!
_________________
The End of the Internet!
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4122
Location: Houston, Republic of Texas

PostPosted: Fri Nov 23, 2012 5:39 pm    Post subject: Reply with quote

FWIW - this isnt exclusive to lvm, btrfs subvolumes, or anything of the sort

i have a pretty boring vanilla setup

Code:

/dev/sda1   *          63   327682047   163840992+   7  HPFS/NTFS/exFAT
/dev/sda2   *   327682048   327761919       39936   83  Linux
/dev/sda3       327761920   331970559     2104320   83  Linux
/dev/sda4       331970560   625141759   146585600   83  Linux
fdisk: cannot open /dev/mapper/root: No such file or directory


/dev/sda2 is boot (actually, i dont use that any more, as i have /boot on a thumb drive)
/dev/sda3 is swap (well, crypt-swap)
/dev/sda4 is root - i have no separate partitions, just everything on root (i had to resize my drives, TLDR, this was the only way to minimize downtime while doing a resize during a workday)

anyway, yall arent alone, and it isnt just your lvm/btrfs that's causing the issue.

All i have is:
Code:

hplaptop ~ # ls -alh /dev/mapper
total 0
drwxr-xr-x  2 root root       80 Nov 22 20:09 .
drwxr-xr-x 10 root root     5.9K Nov 22 20:09 ..
crw-------  1 root root  10, 236 Nov 22 20:09 control
brw-------  1 root root 253,   1 Nov 22 20:09 crypt-swap


aside from a few unsightly errors, it has made no *tangible* difference. But I wouldn't be able to fsck my stuff, so that could present an issue.

The one wrinkle for me - I don't use udev. At all. I use mdev,

Code:

alsasound |default
bootmisc | boot
consolefont | boot
dbus |default
devfs |sysinit
dmcrypt | boot default
dmesg |sysinit
fbcondecor |sysinit
fsck | boot
hostname | boot
hwclock | boot
keymaps | boot
killprocs |shutdown
local |default nonetwork
localmount | boot
mdev |sysinit
modules | boot
mount-ro |shutdown
mtab | boot
net.lo | boot
net.wlan0 |default
ntpd |default
procfs | boot
root | boot
savecache |shutdown
sshd |default
swap | boot
swapfiles | boot
sysctl | boot
syslog-ng |default
termencoding | boot
urandom | boot
vixie-cron |default
xdm |default


so mine may be a bit more self-inflicted. I think we're hitting a bug of some sort here, but I'm not sure quite what.

I may go the route of the "mini" init script above. It's just not pressing enough right now for me to really fiddle with it.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
truc
Advocate
Advocate


Joined: 25 Jul 2005
Posts: 3199

PostPosted: Fri Nov 23, 2012 5:55 pm    Post subject: Reply with quote

you can try from the cli:
does
Code:
dmsetup table
output something for you? what about
Code:
dmsetup info
?. if that's the case, just issue dmsetup mknodes and see how it goes.
_________________
The End of the Internet!
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4122
Location: Houston, Republic of Texas

PostPosted: Fri Nov 23, 2012 9:25 pm    Post subject: Reply with quote

truc wrote:
you can try from the cli:
does
Code:
dmsetup table
output something for you? what about
Code:
dmsetup info
?. if that's the case, just issue dmsetup mknodes and see how it goes.


aye, those are good to go
dmsetup mknodes does create the device nodes under /dev/mapper

since a)nothing is functionally wrong, and b)dmsetup mknodes is all i really need, i think ill probably go your route of just rolling my own init script to do the job

not too worried for me really, but somewhere in this must exist a bug worth filing - i just dont know *where* precisely, or rather, i dont know what component isnt doing its job.
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum