iptables failure with filters that are compiled as modules

Message

Shadow_Fury · Post by **Shadow_Fury** » Fri Nov 25, 2022 8:29 pm

as said in the title, whenever i try to start the docker daemon with

rc-service docker start

iptables fails.

here is the iptables command in question:

Code: Select all

iptables --wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER

.

the relevant error (reproduced when trying to run iptables from terminal):

Code: Select all

Warning: Extension addrtype revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.

any help is appreciated.

full docker log:

Code: Select all

time="2022-11-25T20:06:46.288769827Z" level=info msg="Starting up"
time="2022-11-25T20:06:46.289691358Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2022-11-25T20:06:46.289705771Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2022-11-25T20:06:46.289720164Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2022-11-25T20:06:46.289728333Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2022-11-25T20:06:46.290768613Z" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2022-11-25T20:06:46.290781776Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2022-11-25T20:06:46.290791324Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2022-11-25T20:06:46.290798222Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2022-11-25T20:06:46.291329704Z" level=error msg="Failed to built-in GetDriver graph btrfs /var/lib/docker"
time="2022-11-25T20:06:46.291767993Z" level=error msg="failed to mount overlay: no such device" storage-driver=overlay2
time="2022-11-25T20:06:46.291833756Z" level=error msg="exec: \"fuse-overlayfs\": executable file not found in $PATH" storage-driver=fuse-overlayfs
time="2022-11-25T20:06:46.291841594Z" level=error msg="Failed to built-in GetDriver graph aufs /var/lib/docker"
time="2022-11-25T20:06:46.291846671Z" level=error msg="Failed to built-in GetDriver graph overlay /var/lib/docker"
time="2022-11-25T20:06:46.291851216Z" level=error msg="Failed to built-in GetDriver graph devicemapper /var/lib/docker"
time="2022-11-25T20:06:46.301745796Z" level=warning msg="Your kernel does not support cgroup blkio weight"
time="2022-11-25T20:06:46.301757356Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
time="2022-11-25T20:06:46.301839677Z" level=info msg="Loading containers: start."
time="2022-11-25T20:06:46.328349122Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2022-11-25T20:06:46.336185642Z" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
failed to start daemon: Error initializing network controller: Error creating default "bridge" network: Failed to program NAT chain: Failed to inject DOCKER in PREROUTING chain: iptables failed: iptables --wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER: Warning: Extension addrtype revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.
 (exit status 1)

Post by Hu » Fri Nov 25, 2022 8:46 pm

The error suggests that addrtype support is missing. Is it? What is the output of uname -r; zgrep NETFILTER /proc/config.gz?

Shadow_Fury · Post by **Shadow_Fury** » Fri Nov 25, 2022 8:52 pm

Hu wrote:The error suggests that addrtype support is missing. Is it? What is the output of uname -r; zgrep NETFILTER /proc/config.gz?

Code: Select all

6.0.3-gentoo
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_BRIDGE_NETFILTER=m
CONFIG_NETFILTER_INGRESS=y
CONFIG_NETFILTER_EGRESS=y
CONFIG_NETFILTER_SKIP_EGRESS=y
CONFIG_NETFILTER_NETLINK=y
CONFIG_NETFILTER_FAMILY_BRIDGE=y
CONFIG_NETFILTER_FAMILY_ARP=y
CONFIG_NETFILTER_NETLINK_ACCT=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=y
CONFIG_NETFILTER_NETLINK_OSF=m
CONFIG_NETFILTER_CONNCOUNT=m
# CONFIG_NETFILTER_NETLINK_GLUE_CT is not set
CONFIG_NETFILTER_SYNPROXY=m
CONFIG_NETFILTER_XTABLES=y
CONFIG_NETFILTER_XTABLES_COMPAT=y
CONFIG_NETFILTER_XT_MARK=m
CONFIG_NETFILTER_XT_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_AUDIT=m
CONFIG_NETFILTER_XT_TARGET_CHECKSUM=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_HL=m
CONFIG_NETFILTER_XT_TARGET_HMARK=m
CONFIG_NETFILTER_XT_TARGET_IDLETIMER=m
CONFIG_NETFILTER_XT_TARGET_LED=m
CONFIG_NETFILTER_XT_TARGET_LOG=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_NAT=m
CONFIG_NETFILTER_XT_TARGET_NETMAP=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=y
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_RATEEST=m
CONFIG_NETFILTER_XT_TARGET_REDIRECT=m
CONFIG_NETFILTER_XT_TARGET_MASQUERADE=m
CONFIG_NETFILTER_XT_TARGET_TEE=m
CONFIG_NETFILTER_XT_TARGET_TPROXY=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=y
CONFIG_NETFILTER_XT_TARGET_TCPMSS=y
CONFIG_NETFILTER_XT_TARGET_TCPOPTSTRIP=m
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_NETFILTER_XT_MATCH_BPF=m
CONFIG_NETFILTER_XT_MATCH_CGROUP=m
CONFIG_NETFILTER_XT_MATCH_CLUSTER=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNLABEL=m
CONFIG_NETFILTER_XT_MATCH_CONNLIMIT=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_CPU=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DEVGROUP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ECN=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_HL=m
CONFIG_NETFILTER_XT_MATCH_IPCOMP=m
CONFIG_NETFILTER_XT_MATCH_IPRANGE=m
CONFIG_NETFILTER_XT_MATCH_IPVS=m
CONFIG_NETFILTER_XT_MATCH_L2TP=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_NFACCT=m
CONFIG_NETFILTER_XT_MATCH_OSF=m
CONFIG_NETFILTER_XT_MATCH_OWNER=m
CONFIG_NETFILTER_XT_MATCH_POLICY=y
CONFIG_NETFILTER_XT_MATCH_PHYSDEV=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_RATEEST=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_RECENT=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_SOCKET=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_TIME=m
CONFIG_NETFILTER_XT_MATCH_U32=m
CONFIG_SECURITY_SMACK_NETFILTER=y

also, when gradually removing parts of the command, the error disappears when

Code: Select all

--dst-type LOCAL

is removed

Shadow_Fury · Post by **Shadow_Fury** » Fri Nov 25, 2022 9:37 pm

vmm is also failing:

Code: Select all

iptables -w --table filter --insert LIBVIRT_FWI --destination 192.168.122.0/24 --out-interface virbr0 --match conntrack --ctstate ESTABLISHED,RELATED --jump ACCEPT: Warning: Extension conntrack revision 0 not supported, missing kernel module?
iptables: No chain/target/match by that name.

it used to work, until i started to follow the gentoo wiki's instructions on enabling the docker required kernel modules, as well as enabling the modules listed by /usr/share/docker/contrib/check-config.sh:

Code: Select all

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled (as module)
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_NETFILTER_XT_MARK: enabled (as module)
- CONFIG_IP_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: enabled
    (cgroup swap accounting is currently enabled)
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_CGROUP_PERF: enabled
- CONFIG_CGROUP_HUGETLB: enabled
- CONFIG_NET_CLS_CGROUP: enabled (as module)
- CONFIG_CGROUP_NET_PRIO: enabled
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_NF_TARGET_REDIRECT: enabled (as module)
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_PROTO_TCP: enabled
- CONFIG_IP_VS_PROTO_UDP: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_EXT3_FS: enabled
- CONFIG_EXT3_FS_XATTR: missing
- CONFIG_EXT3_FS_POSIX_ACL: enabled
- CONFIG_EXT3_FS_SECURITY: enabled
    (enable these ext3 configs if you are using ext3 as backing filesystem)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: enabled
- CONFIG_EXT4_FS_SECURITY: enabled
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: enabled (as module)
    - CONFIG_BRIDGE_VLAN_FILTERING: enabled
      Optional (for encrypted networks):
      - CONFIG_CRYPTO: enabled
      - CONFIG_CRYPTO_AEAD: enabled
      - CONFIG_CRYPTO_GCM: enabled
      - CONFIG_CRYPTO_SEQIV: enabled
      - CONFIG_CRYPTO_GHASH: enabled
      - CONFIG_XFRM: enabled
      - CONFIG_XFRM_USER: enabled
      - CONFIG_XFRM_ALGO: enabled
      - CONFIG_INET_ESP: enabled (as module)
  - "ipvlan":
    - CONFIG_IPVLAN: enabled (as module)
  - "macvlan":
    - CONFIG_MACVLAN: enabled (as module)
    - CONFIG_DUMMY: enabled (as module)
  - "ftp,tftp client in container":
    - CONFIG_NF_NAT_FTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
    - CONFIG_NF_NAT_TFTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: missing
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled (as module)
    - CONFIG_BTRFS_FS_POSIX_ACL: enabled
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled
    - CONFIG_DM_THIN_PROVISIONING: enabled (as module)
  - "overlay":
    - CONFIG_OVERLAY_FS: enabled (as module)
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000

Post by **pietinger** » Fri Nov 25, 2022 10:05 pm

Do you have this in your kernel configuration ? Enable in BOTH submenus all options with <M>:

Code: Select all

--- Network packet filtering framework (Netfilter)
[*]   Advanced netfilter configuration
      Core Netfilter Configuration  --->
[*]   IP set support  --->
...
      IP: Netfilter Configuration  --->
[*]   IPv4/IPV6 bridge connection tracking support

Maybe you are only missing CONFIG_NF_CONNTRACK. After starting docker you can use "lsmod" to determine which modules you actually need.

Shadow_Fury · Post by **Shadow_Fury** » Fri Nov 25, 2022 10:37 pm

pietinger wrote:Do you have this in your kernel configuration ? Enable in BOTH submenus all options with <M>:
Code: Select all
--- Network packet filtering framework (Netfilter)
[*]   Advanced netfilter configuration
      Core Netfilter Configuration  --->
[*]   IP set support  --->
...
      IP: Netfilter Configuration  --->
[*]   IPv4/IPV6 bridge connection tracking support
Maybe you are only missing CONFIG_NF_CONNTRACK. After starting docker you can use "lsmod" to determine which modules you actually need.

all mentioned menu items are enabled, docker still crashes with the same error, and vmm is still failing (both on iptables)

Shadow_Fury · Post by **Shadow_Fury** » Sun Dec 04, 2022 5:43 pm

found the issue, though not closing it, since it seems like this is a bug...

Apparently, any iptables things that you want to work HAVE to be compiled into the kernel, compiling to modules doesn't work, even when later using modprobe to load them.

Post by **pietinger** » Mon Dec 05, 2022 6:35 pm

Shadow_Fury wrote:[...] Apparently, any iptables things that you want to work HAVE to be compiled into the kernel, compiling to modules doesn't work, even when later using modprobe to load them.

Can you tell us if it was only one (or some) special options/modules, or if you had to include all static into the kernel (would look to me very strange because you can use modular iptables modules) ?

szatox · Post by **szatox** » Mon Dec 05, 2022 9:40 pm

I have a bunch of things regarding iptables built as modules and it works fine, though I had to load them manually before the first use.
After that, they get autoloaded... I suppose there might be some clever udev trick keeping it working after reboot... Haven't bothered to investigate this though, been busy wasting time on other things.

Shadow_Fury · Post by **Shadow_Fury** » Sun Dec 11, 2022 5:06 am

pietinger wrote:
Shadow_Fury wrote:[...] Apparently, any iptables things that you want to work HAVE to be compiled into the kernel, compiling to modules doesn't work, even when later using modprobe to load them.
Can you tell us if it was only one (or some) special options/modules, or if you had to include all static into the kernel (would look to me very strange because you can use modular iptables modules) ?

as far as i can tell, any filter module needs to be compiled into the kernel. when i moved one that had been working to a module, it broke VMM, and when i compiled the requisite modules into the kernel, both it and docker started working fine