I have run into a similar problem in the context of creating a Gentoo VM in Xen running on Gentoo. I'm posting to this topic because a search for "dolvm invoked, but LVM binary not available" provides very few results, so this looks to be an unusual error condition.
Background:
I have had a Gentoo Xen hypervisor running Gentoo VMs for over a year with no problem assigning LVM logical volumes for various VMs. My Xen configuration files for a Gentoo VM environment had been working fine. Then I started having problems with Qemu in trying to build a Raspberry Pi 64 image (chrooting in the VM to aarch64) so I upgraded the kernel, xen and xentools. Now I cannot launch a VM for Gentoo because LVM is not found and/or working in the guest start-up.
My kernel is used both by Dom0 [hermes] and the Gentoo VMs.
Code: Select all
hermes /etc/xen # uname -a
Linux hermes 5.15.80-gentoo-x86_64 #1 SMP Sun Jan 8 17:40:29 PST 2023 x86_64 AMD FX(tm)-8120 Eight-Core Processor AuthenticAMD GNU/Linux
hermes /etc/xen #
I use LVM and in DOM0 the group, /dev/vg0, appears and the /dev/mapper has all the names appearing. However, when I try to launch a new Gentoo VM, the assignment of an LVM logical volume no longer works. I altered by kernel command line and found if I place "dolvm" immediately after the volume, I get a more descriptive error message. Earlier I was just getting a message to the effect the drive could not be found and no mention of lvm. At least now, a warning "LVM binary not available" gives me further insight into the problem.
Here's the error message from an "xl create gentoo1.conf -c" command:
Code: Select all
[ 3.267850] Write protecting the kernel read-only data: 22528k
[ 3.282598] Freeing unused kernel image (text/rodata gap) memory: 2036K
[ 3.283555] Freeing unused kernel image (rodata/data gap) memory: 1072K
[ 3.597873] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 3.597898] rodata_test: all tests were successful
[ 3.597910] Run /init as init process
>> Genkernel 4.2.6 (2023-01-09 02:34:08 UTC). Linux kernel 5.15.80-gentoo-x86_64
>> Activating udev ...
!! dolvm invoked but LVM binary not available; Skipping LVM volume group activation ...
>> Determining root device (trying /dev/xvda3) ........
!! Block device /dev/xvda3 is not a valid root device ...
!! Could not find the root block device in /dev/xvda3.
!! Please specify another value or:
!! - press Enter for the same
!! - type "shell" for a shell
!! - type "q" to skip ...
root block device(/dev/xvda3) ::
I end up returning to the DomO console and then executing an "xl destroy [ID]".
I've entered the BusyBox shell and searched /dev and nothing shows up, so the LVM maps are not present.
My Xen packages:
Code: Select all
hermes /etc/xen # date; eix -I xen
Tue Jan 10 16:55:18 PST 2023
[I] app-emulation/xen
Available versions: 4.15.4_pre1^st ~4.15.4_pre2^st 4.15.4^st ~4.16.3_pre^st ~4.16.3_pre1^st {+boot-symlinks debug efi flask}
Installed versions: 4.15.4^st(19:22:28 01/08/23)(-debug -efi -flask)
Homepage: https://xenproject.org
Description: The Xen virtual machine monitor
[I] app-emulation/xen-tools
Available versions: 4.15.4_pre1-r1(0/4.15)^t ~4.15.4_pre2-r1(0/4.15)^t 4.15.4-r1(0/4.15)^t ~4.16.3_pre-r1(0/4.16)^t ~4.16.3_pre1-r1(0/4.16)^t {api debug doc +hvm +ipxe lzma ocaml ovmf pygrub python +qemu +qemu-traditional +rombios screen sdl selinux static-libs system-ipxe system-qemu system-seabios systemd zstd PYTHON_SINGLE_TARGET="python3_8 python3_9 python3_10 python3_11"}
Installed versions: 4.15.4-r1(0/4.15)^t(19:34:15 01/08/23)(hvm ipxe qemu qemu-traditional rombios sdl -api -debug -doc -lzma -ocaml -ovmf -pygrub -python -screen -selinux -static-libs -system-ipxe -system-qemu -system-seabios PYTHON_SINGLE_TARGET="python3_10 -python3_8 -python3_9")
Homepage: https://xenproject.org
Description: Xen tools including QEMU and xl
Found 2 matches
hermes /etc/xen #
Sample VM configuration file:
Code: Select all
hermes /etc/xen # cat gentoo1.conf
#
# 1/14/2022 jlpoole first VM for gentoo1
# based on https://wiki.gentoo.org/wiki/Xen#Networking_on_domains
#
kernel = "/boot/vmlinuz-5.15.80-gentoo-x86_64"
ramdisk = "/boot/initramfs-5.15.80-gentoo-x86_64.img"
#bootargs = "dolvm"
#bootargs = "root=/dev/sda3"
#memory = 16384
memory = 8192
vcpus = 2
name = "gentoo1"
#disk = ['/dev/mapper/vg0-gentoo1,xvda3,w'] # Map the disk image to the virtual /dev/sda3
disk = ['/dev/mapper/vg0-gentoo1,,xvda3,w'] # 1/9/2023 above was not working, so try this
root = "/dev/xvda3 ro dolvm"
#
# 1/10/23 cmdline supplant root command
#
#cmdline = "/dev/xvda3 dolvm ro"
#vif = ['bridge=br0'] # set the bridge name to br0 (see later), otherwise xen will use xenbr0
#
# Hermes has [redacted]
# [redacted]
#
vif = ['mac=[redacted],bridge=xenbr0']
hermes /etc/xen #
Activating '#bootargs = "dolvm"' does not change anything.
Here's what I have for lvm:
Code: Select all
hermes /etc/xen # eix -I lvm
[I] sys-devel/llvm
Available versions:
(13) 13.0.1^t
(14) 14.0.6-r2^t
(15) 15.0.5(15/15)^t 15.0.6(15/15)^t ~15.0.6-r1(15/15)^t **15.0.6.9999(15/15)*l^t
(16) ~*16.0.0_pre20221217(16/16.0.0_pre20221217)^t ~*16.0.0_pre20221226(16/16.0.0_pre20221226)^t ~*16.0.0_pre20230101(16/16.0.0_pre20230101)^t **16.0.0.9999(16/16.0.0.9999)*l^t
{+binutils-plugin debug doc exegesis libedit +libffi ncurses test verify-sig xar xml z3 zstd ABI_MIPS="n32 n64 o32" ABI_S390="32 64" ABI_X86="32 64 x32" LLVM_TARGETS="AArch64 AMDGPU ARC ARM AVR BPF CSKY DirectX Hexagon Lanai LoongArch M68k MSP430 Mips NVPTX PowerPC RISCV SPIRV Sparc SystemZ VE WebAssembly X86 XCore Xtensa"}
Installed versions: 15.0.6(15/15)^t(20:37:25 12/23/22)(binutils-plugin libffi ncurses xml -debug -doc -exegesis -libedit -test -verify-sig -xar -z3 -zstd ABI_MIPS="-n32 -n64 -o32" ABI_S390="-32 -64" ABI_X86="64 -32 -x32" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon Lanai MSP430 Mips NVPTX PowerPC RISCV Sparc SystemZ VE WebAssembly X86 XCore -ARC -CSKY -DirectX -LoongArch -M68k -SPIRV")
Homepage: https://llvm.org/
Description: Low Level Virtual Machine
[I] sys-devel/llvm-common
Available versions: 13.0.1 14.0.6 15.0.5 15.0.6 **15.0.6.9999*l ~*16.0.0_pre20221217 **16.0.0_pre20221226 ~*16.0.0_pre20230101 **16.0.0.9999*l {verify-sig}
Installed versions: 15.0.6(12:56:22 12/23/22)(-verify-sig)
Homepage: https://llvm.org/
Description: Common files shared between multiple slots of LLVM
[I] sys-devel/llvm-toolchain-symlinks
Available versions:
(14) ~14-r1
(15) 15-r1
(16) ~*16-r1
{multilib-symlinks +native-symlinks}
Installed versions: 15-r1(15)(20:37:47 12/23/22)(native-symlinks -multilib-symlinks)
Homepage: https://wiki.gentoo.org/wiki/Project:LLVM
Description: Symlinks to use LLVM on binutils-free system
[I] sys-devel/llvmgold
Available versions: 13-r2 14 15 ~*16
Installed versions: 15(20:38:00 12/23/22)
Homepage: https://llvm.org/
Description: LLVMgold plugin symlink for autoloading
[I] sys-fs/lvm2
Available versions: 2.03.14-r5 2.03.17-r1 {lvm lvm2create-initrd readline sanlock selinux static static-libs systemd thin +udev}
Installed versions: 2.03.17-r1(13:03:23 01/07/23)(lvm lvm2create-initrd readline udev -sanlock -selinux -static -static-libs -systemd -thin)
Homepage: https://sourceware.org/lvm2/
Description: User-land utilities for LVM2 (device-mapper) software
Found 5 matches
hermes /etc/xen #
One thing I've noticed, I have other servers on my network which use LVM and at times I've seen on a machine some suggestion that the LVM from another server is being communicated with or tied in. I've not tried to have the LVM traverse multiple machines, but maybe I have that occurring, if it can and the "sharing" is somehow at play?