I heve been running LXD for a while now, using it mostly to run linux virtual machines. It was working flawlessly, without any issue whatsoever. However, I decided to try also docker, for using it for isolating specific applications. It was not hard to setup docker, but LXD stopped working as soon as I finish it. This is what happens now when I run LXD:
Code: Select all
~> sudo rc-service lxd start
Senha:
* Starting lxcfs. ... [ ok ]
* Starting lxd service ... [ ok ]
~> lxc list
+------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
| ubuntu-lts | RUNNING | 10.212.105.243 (eth0) | fd42:daae:1bcc:36cb:216:3eff:fe8a:9e79 (eth0) | VIRTUAL-MACHINE | 0 |
+------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
~> lxc list
+------------+---------+------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------------+---------+------+------+-----------------+-----------+
| ubuntu-lts | STOPPED | | | VIRTUAL-MACHINE | 0 |
+------------+---------+------+------+-----------------+-----------+
~> lxc list
+------------+---------+------+------+-----------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------------+---------+------+------+-----------------+-----------+
| ubuntu-lts | STOPPED | | | VIRTUAL-MACHINE | 0 |
+------------+---------+------+------+-----------------+-----------+
~> dmesg | grep lxc
[ 67.298021] audit: type=1400 audit(1643299870.396:319): avc: denied { write } for pid=4244 comm="lxc" name="unix.socket" dev="dm-1" ino=26235768 scontext=staff_u:staff_r:staff_t tcontext=staff_u:object_r:var_lib_t tclass=sock_file permissive=1
~> lxc start ubuntu-lts
Error: Failed to run: forklimits limit=memlock:unlimited:unlimited -- /usr/bin/qemu-system-x86_64 -S -name ubuntu-lts -uuid 5de2f6da-f44b-4648-a1f3-266cf3306b59 -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/log/lxd/ubuntu-lts/qemu.conf -spice unix=on,disable-ticketing=on,addr=/var/log/lxd/ubuntu-lts/qemu.spice -pidfile /var/log/lxd/ubuntu-lts/qemu.pid -D /var/log/lxd/ubuntu-lts/qemu.log -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas nobody: char device redirected to /dev/pts/2 (label console)
: Process exited with non-zero value 1
Try `lxc info --show-log ubuntu-lts` for more info
~> lxc info --show-log ubuntu-lts
Name: ubuntu-lts
Location: none
Remote: unix://
Arquitetura: x86_64
Criado: 2022/01/18 22:22 -03
Status: Stopped
Type: virtual-machine
Profiles: default
Log:
qemu-system-x86_64:/var/log/lxd/ubuntu-lts/qemu.conf:128: vhost-vsock: unable to set guest cid: Address already in use
~>
I have google a little bit and found this bug report, which shows some similarities with the issue I am reporting. However, the bug seems to have been fixed, so it is hardly what is happening to me.
Those are the details of my install of LXD, LXC, QEMU and DOCKER:
Code: Select all
~> groups
lp wheel cron audio docker kvm users vboxusers pcap lxd vinicius
~> equery uses lxd
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for app-containers/lxd-4.0.8-r2:
U I
+ + ipv6 : Add support for IP version 6
+ + nls : Add Native Language Support (using gettext - GNU locale utilities)
+ + verify-sig : Verify upstream signatures on distfiles
~> equery uses lxc
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for app-containers/lxc-4.0.11_p1-r1:
U I
- - doc : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of
globally
- - io-uring : Enable io_uring support, and use io_uring instead of epoll
- - man : Build and install man pages
+ + pam : Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip
+ + seccomp : Enable seccomp (secure computing mode) to perform system call filtering at runtime to increase
security of programs
+ + ssl : Add support for SSL/TLS connections (Secure Socket Layer / Transport Layer Security)
+ + tools : Build and install additional command line tools
+ + verify-sig : Verify upstream signatures on distfiles
~> equery uses qemu
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for app-emulation/qemu-6.2.0:
U I
- - accessibility : Adds support for braille displays using brltty
+ + aio : Enables support for Linux's Async IO
+ + alsa : Enable alsa output for sound emulation
- - bpf : Enable eBPF support for RSS implementation.
+ + bzip2 : Use the bzlib compression library
+ + curl : Support ISOs / -cdrom directives via HTTP or HTTPS.
- - debug : Enable extra debug codepaths, like asserts and extra output. If you want
to get meaningful backtraces see
https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces
+ + doc : Add extra documentation (API, Javadoc, etc). It is recommended to enable
per package instead of globally
+ + fdt : Enables firmware device tree support
+ + filecaps : Use Linux file capabilities to control privilege rather than set*id (this
is orthogonal to USE=caps which uses capabilities at runtime e.g. libcap)
- - fuse : Enables FUSE block device export
- - glusterfs : Enables GlusterFS cluster fileystem via sys-cluster/glusterfs
- - gnutls : Enable TLS support for the VNC console server. For 1.4 and newer this also
enables WebSocket support. For 2.0 through 2.3 also enables disk quorum
support.
- - gtk : Add support for x11-libs/gtk+ (The GIMP Toolkit)
- - infiniband : Enable Infiniband RDMA transport support
- - io-uring : Enable efficient I/O via sys-libs/liburing.
- - iscsi : Enable direct iSCSI support via net-libs/libiscsi instead of indirectly
via the Linux block layer that sys-block/open-iscsi does.
- - jack : Add support for the JACK Audio Connection Kit
- - jemalloc : Enable jemalloc allocator support
+ + jpeg : Enable jpeg image support for the VNC console server
- - lzo : Enable support for lzo compression
- - multipath : Enable multipath persistent reservation passthrough via
sys-fs/multipath-tools.
+ + ncurses : Enable the ncurses-based console
- - nfs : Enable NFS support
+ + nls : Add Native Language Support (using gettext - GNU locale utilities)
- - numa : Enable NUMA support
- - opengl : Add support for OpenGL (3D graphics)
+ + oss : Add support for OSS (Open Sound System)
+ + pin-upstream-blobs : Pin the versions of BIOS firmware to the version included in the upstream
release. This is needed to sanely support
migration/suspend/resume/snapshotting/etc... of instances. When the blobs
are different, random corruption/bugs/crashes/etc... may be observed.
- - plugins : Enable qemu plugin API via shared library loading.
+ + png : Enable png image support for the VNC console server
+ + pulseaudio : Enable pulseaudio output for sound emulation
- - python : Add optional support/bindings for the Python language
- - python_targets_python3_10 : Build with Python 3.10
- - python_targets_python3_8 : Build with Python 3.8
+ + python_targets_python3_9 : Build with Python 3.9
- - qemu_softmmu_targets_aarch64 : ARM64 system emulation target
- - qemu_softmmu_targets_alpha : system emulation target
+ + qemu_softmmu_targets_arm : system emulation target
- - qemu_softmmu_targets_avr : system emulation target
- - qemu_softmmu_targets_cris : system emulation target
- - qemu_softmmu_targets_hppa : system emulation target
- - qemu_softmmu_targets_i386 : system emulation target
- - qemu_softmmu_targets_m68k : system emulation target
- - qemu_softmmu_targets_microblaze : system emulation target
- - qemu_softmmu_targets_microblazeel : system emulation target
- - qemu_softmmu_targets_mips : system emulation target
- - qemu_softmmu_targets_mips64 : system emulation target
- - qemu_softmmu_targets_mips64el : system emulation target
- - qemu_softmmu_targets_mipsel : system emulation target
- - qemu_softmmu_targets_nios2 : system emulation target
- - qemu_softmmu_targets_or1k : OpenRISC system emulation target
- - qemu_softmmu_targets_ppc : system emulation target
- - qemu_softmmu_targets_ppc64 : system emulation target
- - qemu_softmmu_targets_riscv32 : system emulation target
- - qemu_softmmu_targets_riscv64 : system emulation target
- - qemu_softmmu_targets_rx : Renesas RX system emulation target
- - qemu_softmmu_targets_s390x : system emulation target
- - qemu_softmmu_targets_sh4 : system emulation target
- - qemu_softmmu_targets_sh4eb : system emulation target
+ + qemu_softmmu_targets_sparc : system emulation target
- - qemu_softmmu_targets_sparc64 : system emulation target
- - qemu_softmmu_targets_tricore : system emulation target
+ + qemu_softmmu_targets_x86_64 : system emulation target
- - qemu_softmmu_targets_xtensa : system emulation target
- - qemu_softmmu_targets_xtensaeb : system emulation target
- - qemu_user_targets_aarch64 : ARM64 userspace emulation target
- - qemu_user_targets_aarch64_be : ARM64 big endian userspace emulation target
- - qemu_user_targets_alpha : userspace emulation target
- - qemu_user_targets_arm : ARM (little endian) userspace emulation target
- - qemu_user_targets_armeb : ARM (big endian) userspace emulation target
- - qemu_user_targets_cris : userspace emulation target
- - qemu_user_targets_hexagon : Qualcomm hexagon userspace emulation target
- - qemu_user_targets_hppa : userspace emulation target
- - qemu_user_targets_i386 : userspace emulation target
- - qemu_user_targets_m68k : userspace emulation target
- - qemu_user_targets_microblaze : userspace emulation target
- - qemu_user_targets_microblazeel : userspace emulation target
- - qemu_user_targets_mips : userspace emulation target
- - qemu_user_targets_mips64 : userspace emulation target
- - qemu_user_targets_mips64el : userspace emulation target
- - qemu_user_targets_mipsel : userspace emulation target
- - qemu_user_targets_mipsn32 : MIPS N32 ABI userspace emulation target
- - qemu_user_targets_mipsn32el : MIPS N32 ABI LE userspace emulation target
- - qemu_user_targets_nios2 : userspace emulation target
- - qemu_user_targets_or1k : OpenRISC system emulation target
- - qemu_user_targets_ppc : PowerPC 32-bit big endian userspace emulation target
- - qemu_user_targets_ppc64 : PowerPC 64-bit big endian userspace emulation target
- - qemu_user_targets_ppc64abi32 : userspace emulation target
- - qemu_user_targets_ppc64le : PowerPC 64-bit little endian userspace emulation target
- - qemu_user_targets_riscv32 : userspace emulation target
- - qemu_user_targets_riscv64 : userspace emulation target
- - qemu_user_targets_s390x : userspace emulation target
- - qemu_user_targets_sh4 : userspace emulation target
- - qemu_user_targets_sh4eb : userspace emulation target
- - qemu_user_targets_sparc : userspace emulation target
- - qemu_user_targets_sparc32plus : userspace emulation target
- - qemu_user_targets_sparc64 : userspace emulation target
+ + qemu_user_targets_x86_64 : userspace emulation target
- - qemu_user_targets_xtensa : userspace emulation target
- - qemu_user_targets_xtensaeb : userspace emulation target
- - rbd : Enable rados block device backend support, see
https://docs.ceph.com/en/mimic/rbd/qemu-rbd/
- - sasl : Add support for the Simple Authentication and Security Layer
- - sdl : Enable the SDL-based console
- - sdl-image : SDL Image support for icons
+ + seccomp : Enable seccomp (secure computing mode) to perform system call filtering at
runtime to increase security of programs
+ + slirp : Enable TCP/IP in hypervisor via net-libs/libslirp
- - smartcard : Enable smartcard support
- - snappy : Enable support for Snappy compression (as implemented in app-arch/snappy)
+ + spice : Enable Spice protocol support via app-emulation/spice
- - ssh : Enable SSH based block device support via net-libs/libssh2
- - static : Build the User and Software MMU (system) targets as well as tools as
static binaries
- - static-user : Build the User targets as static binaries
- - systemtap : Enable SystemTAP/DTrace tracing
- - test : Enable dependencies and/or preparations necessary to run tests (usually
controlled by FEATURES=test but can be toggled independently)
+ + udev : Enable virtual/udev integration (device discovery, power and storage
device support, etc)
- - usb : Enable USB passthrough via dev-libs/libusb
+ + usbredir : Use sys-apps/usbredir to redirect USB devices to another machine over TCP
- - vde : Enable VDE-based networking
+ + vhost-net : Enable accelerated networking using vhost-net, see
https://www.linux-kvm.org/page/VhostNet
- - vhost-user-fs : Enable shared file system access using the FUSE protocol carried over
virtio.
- - virgl : Enable experimental Virgil 3d (virtual software GPU)
+ + virtfs : Enable VirtFS via virtio-9p-pci / fsdev. See
https://wiki.qemu.org/Documentation/9psetup
+ + vnc : Enable VNC (remote desktop viewer) support
- - vte : Enable terminal support ( x11-libs/vte) in the GTK+ interface
+ + xattr : Add support for getting and setting POSIX extended attributes, through
sys-apps/attr. Requisite for the virtfs backend.
- - xen : Enables support for Xen backends
- - xfs : Support xfsctl() notification and syncing for XFS backed virtual disks.
- - zstd : Enable support for ZSTD compression
~> equery uses docker
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for app-containers/docker-20.10.9:
U I
- - apparmor : Enable AppArmor support.
- - aufs : Enables dependencies for the "aufs" graph driver, including necessary kernel flags.
- - btrfs : Enables dependencies for the "btrfs" graph driver, including necessary kernel flags.
- - cli : This is a temporary use flag which pulls in app-containers/docker-cli, the docker command
line client. This flag is here to assist in the transition to split packages and will be
removed in a future release.
+ + container-init : Makes the a staticly-linked init system tini available inside a container.
- - device-mapper : Enables dependencies for the "devicemapper" graph driver, including necessary kernel flags.
+ + hardened : Activate default security enhancements for toolchain (gcc, glibc, binutils)
- - overlay : Enables dependencies for the "overlay" graph driver, including necessary kernel flags.
+ + seccomp : Enable seccomp (secure computing mode) to perform system call filtering at runtime to
increase security of programs
Code: Select all
~> lxc-checkconfig
LXC version 4.0.11
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
--- Control groups ---
Cgroups: enabled
Cgroup namespace: enabled
Cgroup v1 mount points:
/sys/fs/cgroup/openrc
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu
/sys/fs/cgroup/cpuacct
/sys/fs/cgroup/blkio
/sys/fs/cgroup/memory
/sys/fs/cgroup/devices
/sys/fs/cgroup/freezer
/sys/fs/cgroup/net_cls
/sys/fs/cgroup/perf_event
/sys/fs/cgroup/net_prio
/sys/fs/cgroup/hugetlb
/sys/fs/cgroup/pids
/sys/fs/cgroup/rdma
/sys/fs/cgroup/misc
/sys/fs/cgroup/debug
Cgroup v2 mount points:
/sys/fs/cgroup/unified
Cgroup v1 systemd controller: missing
Cgroup v1 clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled
--- Misc ---
Veth pair device: enabled, not loaded
Macvlan: enabled, not loaded
Vlan: enabled, loaded
Bridges: enabled, not loaded
Advanced netfilter: enabled, not loaded
CONFIG_NF_NAT_IPV4: missing
CONFIG_NF_NAT_IPV6: missing
CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, loaded
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, loaded
FUSE (for use with lxcfs): enabled, not loaded
--- Checkpoint/Restore ---
checkpoint restore: enabled
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: enabled
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: enabled
CONFIG_NETLINK_DIAG: enabled
File capabilities:
Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig
~>
where the only "disable" message is regarding systemd, which is not an issue here since I am running openrc. For docker this is what I have in the kernel:
Code: Select all
~> /usr/share/docker/contrib/check-config.sh
info: reading kernel config from /proc/config.gz ...
Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_NETFILTER_XT_MARK: enabled (as module)
- CONFIG_IP_NF_NAT: enabled
- CONFIG_NF_NAT: enabled
- CONFIG_POSIX_MQUEUE: enabled
Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: enabled
(cgroup swap accounting is currently enabled)
- CONFIG_LEGACY_VSYSCALL_EMULATE: enabled
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_NF_TARGET_REDIRECT: enabled
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_PROTO_TCP: enabled
- CONFIG_IP_VS_PROTO_UDP: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
- "overlay":
- CONFIG_VXLAN: enabled (as module)
- CONFIG_BRIDGE_VLAN_FILTERING: enabled
Optional (for encrypted networks):
- CONFIG_CRYPTO: enabled
- CONFIG_CRYPTO_AEAD: enabled
- CONFIG_CRYPTO_GCM: enabled
- CONFIG_CRYPTO_SEQIV: enabled
- CONFIG_CRYPTO_GHASH: enabled
- CONFIG_XFRM: enabled
- CONFIG_XFRM_USER: enabled
- CONFIG_XFRM_ALGO: enabled
- CONFIG_INET_ESP: enabled (as module)
- "ipvlan":
- CONFIG_IPVLAN: enabled (as module)
- "macvlan":
- CONFIG_MACVLAN: enabled (as module)
- CONFIG_DUMMY: enabled (as module)
- "ftp,tftp client in container":
- CONFIG_NF_NAT_FTP: enabled (as module)
- CONFIG_NF_CONNTRACK_FTP: enabled (as module)
- CONFIG_NF_NAT_TFTP: enabled (as module)
- CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
- "aufs":
- CONFIG_AUFS_FS: missing
- "btrfs":
- CONFIG_BTRFS_FS: enabled
- CONFIG_BTRFS_FS_POSIX_ACL: enabled
- "devicemapper":
- CONFIG_BLK_DEV_DM: enabled
- CONFIG_DM_THIN_PROVISIONING: enabled
- "overlay":
- CONFIG_OVERLAY_FS: enabled
- "zfs":
- /dev/zfs: present
- zfs command: available
- zpool command: available
Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000
~>
Finally, I have also took care of the subuid and subgid issue as reported here:
Code: Select all
~> cat /etc/subuid
root:1000000:1000000000
lxd:1000000:1000000000
vinicius:1001000000:1000000
developer:1002000000:1000000
~> cat /etc/subgid
root:1000000:1000000000
lxd:1000000:1000000000
vinicius:1001000000:1000000
developer:1002000000:1000000
~>
I have no idea where to go, so any help will be greatly welcome since I really need my LXD virtual machine.
Thank you for your attention.
