Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved] nftables kernel module nft_counter
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
Tender
Apprentice
Apprentice


Joined: 05 Nov 2005
Posts: 166

PostPosted: Sun Nov 21, 2021 7:05 pm    Post subject: [solved] nftables kernel module nft_counter Reply with quote

Hello,

I execute my firewall nftable script and I save the resulting rules with /etc/init.d/nftables save, then there are problems at boot, after login:

Code:
midpower4 ~ # rc-status
Runlevel: default
 hdparm                                                                                                                             [  started  ]
 syslog-ng                                                                                                                          [  started  ]
 acpid                                                                                                                                 [  started  ]
 nftables                                                                                                                             [  stopped ]
 


If I re-execute the nft script all the statements with counters throw errors:

Code:
midpower4 ~ # ./nftclose
./nftclose:59:9-15: Error: Could not process rule: No such file or directory
        counter comment "count all packets"
        ^^^^^^^
./nftclose:60:55-61: Error: Could not process rule: No such file or directory
        vlan id 2 meta protocol {0x8863, 0x8864, arp} counter accept comment "vlan 2 accept only modem related packets"
                                                      ^^^^^^^
./nftclose:61:57-63: Error: Could not process rule: No such file or directory
        vlan id 2 tcp sport {http} ip saddr $ip4-02-tim counter accept comment "vlan 2 accept only modem related packets"
                                                        ^^^^^^^
...
etc etc


when this errors occour there is a strage -1 in nft_counter module:

Code:
midpower4 ~ # lsmod |grep nft
nft_chain_nat          16384  0
nf_nat                 49152  1 nft_chain_nat
nft_ct                 20480  0
nf_conntrack          131072  2 nf_nat,nft_ct
nft_log                16384  0
nft_counter            16384  -1
nf_tables             212992  4 nft_ct,nft_log,nft_counter,nft_chain_nat


To avoid this errors I must:

- delete nftables service from default runlevel and reboot

- login, execute a nftables script with a subset of the needed rules, avoiding all the ones that use counters, save the rules, add nftables to default runlevel and reboot

- after login I can execute the complete firewall script without errors

- but if I save them and reboot the problem happens again

!?

Thanks

Code:

midpower4 ~ # nft -v
nftables v1.0.0 (Fearless Fosdick #2)

midpower4 ~ # emerge --info
Portage 3.0.28 (python 3.9.8-final-0, default/linux/amd64/17.1, gcc-11.2.0, glibc-2.33-r7, 5.10.78-gentoo-dist x86_64)
=================================================================
System uname: Linux-5.10.78-gentoo-dist-x86_64-Intel-R-_Celeron-R-_CPU_J3455_@_1.50GHz-with-glibc2.33
KiB Mem:    16048136 total,   5452356 free
KiB Swap:    7812092 total,   7812092 free
Timestamp of repository gentoo: Tue, 16 Nov 2021 11:20:02 +0000
Head commit of repository gentoo: 9e66040a0ed9da569a73663026e362c50a492c3a
sh bash 5.1_p8
ld GNU ld (Gentoo 2.37_p1 p0) 2.37
app-shells/bash:          5.1_p8::gentoo
dev-lang/perl:            5.34.0-r3::gentoo
dev-lang/python:          3.9.8::gentoo
dev-util/cmake:           3.20.5::gentoo
sys-apps/baselayout:      2.7-r3::gentoo
sys-apps/openrc:          0.44.8::gentoo
sys-apps/sandbox:         2.25::gentoo
sys-devel/autoconf:       2.71-r1::gentoo
sys-devel/automake:       1.16.4::gentoo
sys-devel/binutils:       2.37_p1::gentoo
sys-devel/gcc:            11.2.0::gentoo
sys-devel/gcc-config:     2.4::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.10::gentoo (virtual/os-headers)
sys-libs/glibc:           2.33-r7::gentoo
Repositories:

gentoo
    location: /var/db/repos/gentoo
    sync-type: rsync
    sync-uri: rsync://rsync.fr.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-metamanifest: no
    sync-rsync-extra-opts:

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=silvermont -O2 -pipe -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.0/ext-active/ /etc/php/cgi-php8.0/ext-active/ /etc/php/cli-php8.0/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=silvermont -O2 -pipe -fomit-frame-pointer"
DISTDIR="/mnt/distfiles/"
EMERGE_DEFAULT_OPTS="--autounmask=n -j1 --load-average 1.0 --keep-going=n"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-march=silvermont -O2 -pipe -fomit-frame-pointer"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=silvermont -O2 -pipe -fomit-frame-pointer"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="C.UTF8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5 -l4"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 apache2 bzip2 crypt dri iconv initramfs ipv6 libglvnd libtirpc multilib ncurses nls nptl openmp pcre readline seccomp split-usr ssl unicode xattr zlib" ABI_X86="64" ADA_TARGET="gnat_2019" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" RUBY_TARGETS="ruby26 ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RUSTFLAGS


Last edited by Tender on Wed Dec 29, 2021 6:18 pm; edited 3 times in total
Back to top
View user's profile Send private message
Tender
Apprentice
Apprentice


Joined: 05 Nov 2005
Posts: 166

PostPosted: Sun Nov 21, 2021 7:47 pm    Post subject: Reply with quote

More info:

the complete rules include a three thousands set elements (with counters) added by a python script used by some rules to drop packets coming from dangerous internet hosts/nets.

When the script is executed from command line it adds the set elements without problems, but the resulting ruleset, if saved, cannot be loaded at boot.

May be an Out Of Memory (OOM) in the nft_counter kernel module?


Last edited by Tender on Sun Nov 21, 2021 8:29 pm; edited 1 time in total
Back to top
View user's profile Send private message
mike155
Advocate
Advocate


Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany

PostPosted: Sun Nov 21, 2021 8:07 pm    Post subject: Reply with quote

Quote:
the complete rules include a three thousands set elements (with counters) added by a python script used by some rules to drop packets coming from dangerous internet host/nets.

Why do you do that? IP addresses change frequently. Such a script will always be somewhat outdated...

Why don't you change the default rule to "deny all" and allow the connections you want?
Back to top
View user's profile Send private message
Tender
Apprentice
Apprentice


Joined: 05 Nov 2005
Posts: 166

PostPosted: Sun Nov 21, 2021 8:16 pm    Post subject: Reply with quote

More info:

if the complete rules are saved with /etc/init.d/nftables save , with nftables disabled at boot, after login, if I start the service all the rules are loaded and this is the modules status:

Code:
midpower4 ~ # lsmod |grep nft
nft_masq               16384  2
nft_nat                16384  4
nft_chain_nat          16384  4
nf_nat                 49152  3 nft_nat,nft_masq,nft_chain_nat
nft_reject_ipv4        16384  1
nf_reject_ipv4         16384  1 nft_reject_ipv4
nft_reject             16384  1 nft_reject_ipv4
nft_objref             16384  1
nft_ct                 20480  36
nf_conntrack          131072  4 nf_nat,nft_ct,nft_nat,nft_masq
nft_log                16384  11
nft_counter            16384  5929
nf_tables             212992  553 nft_reject_ipv4,nft_ct,nft_log,nft_nat,nft_objref,nft_counter,nft_masq,nft_chain_nat,nft_reject
midpower4 ~ #


@mike155

The script is executed every day, it fetches the refreshed hosts/nets list from https://iplists.firehol.org/files/firehol_level1.netset
Back to top
View user's profile Send private message
jamapii
l33t
l33t


Joined: 16 Sep 2004
Posts: 637

PostPosted: Thu Nov 25, 2021 12:34 am    Post subject: Reply with quote

First, this looks like a kernel bug to me. The number you mentioned should never go below 0 I believe. Maybe update to 5.14 or 5.15 at some time.

Then there seem to be 5929 counters overall. If you don't use iptables-nft and don't need most counters, try removing most of them.

Then, as I understand, the python script does not update the rules, it only updates a set or a few, independently of the rules. Maybe you can separate the set content from the rules, and load the rules, then the set. (This probably means not using "nftables save").


OOM in the kernel module? I think on an allocation failure, it still should not cause negative ref counts, but just fail the operation. Also I think 3000 set elements should not cause OOM. Unless it's a little router with openwrt?
Back to top
View user's profile Send private message
Tender
Apprentice
Apprentice


Joined: 05 Nov 2005
Posts: 166

PostPosted: Fri Nov 26, 2021 7:36 am    Post subject: Reply with quote

@jamapii

thanks for your comment.

When the 5.15 kernel becomes stable I will use it and see if it fixes; at the moment on my home router I use nftables to manage the main firewall rules, while I am playing with the sets and counters to explore their possibilities: I am using counters everywhere but I believe that this setup is lawful and can be useful as a further test. Yes in fact I could populate the sets later, for now my choice is to start nftables by hand in case I have to restart the router : and this is the strange, the complete set of rules is regularly loaded if I launch the script (/etc/init.d/nftables) by hand after boot, while when it is launched automatically in the default runlevel the problem occurs; for this I think the gentoo system may also be involved, some strange bug or interaction in the init scripts. The machine is decent, with a good CPU and 16 GB of RAM almost all available so it has all the resources it needs.
Back to top
View user's profile Send private message
Tender
Apprentice
Apprentice


Joined: 05 Nov 2005
Posts: 166

PostPosted: Mon Dec 27, 2021 9:25 am    Post subject: Reply with quote

Hello, system boot tested with 5.15.11 kernel, it works, but now works with 5.10.78 too…in the meantime a lot of packages have been updated…
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum