Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
ERROR: ipset failed to stop
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
jhon987
Apprentice
Apprentice


Joined: 18 Nov 2013
Posts: 225

PostPosted: Mon Feb 13, 2017 4:18 pm    Post subject: ERROR: ipset failed to stop Reply with quote

Hello guys,

Every time I shut down or reboot my PC openRC gives the following:
Code:
ipset            | * ipset is in use, can't stop
ipset            | * ERROR: ipset failed to stop


It doesn't seem to affect anything however I do find these error messages annoying, anyone knows how can I stop them?

P.S. I have ipset compiled in my kernel (not as a module) maybe it's relevant, either way I'd rather keep it this way.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Mon Feb 13, 2017 5:26 pm    Post subject: Reply with quote

jhon987 ...

do you have a iptables/firewall rule that references ipset? If so then you probably want the following:

/etc/rc.conf:
rc_ipset_before="firewall"

... or

/etc/conf.d/ipset:
rc_before="firewall"

I had the same issue (some time back) and resolved it with the above.

HTH & best ... khay
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Mon Feb 13, 2017 7:00 pm    Post subject: Reply with quote

Some people may use the iptables service instead of the firewall service, and I gather there are few other options for firewall. The principle is the same though. ipsets have to be built before the rule inserting them to the firewall is called (either iptables-restore, or iptables -I or -A a rule); and an ipset can't be erased until the iptables rule using the set is deleted.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Mon Feb 13, 2017 8:38 pm    Post subject: Reply with quote

cboldt wrote:
Some people may use the iptables service instead of the firewall service, and I gather there are few other options for firewall.

cboldt ... the service is provided by a number of packages (with 'provide firewall' in the init.d script), I expected iptables would also provide this, but I see it doesn't (go figure). If that is the case then the OP can substitute 'firewall' for 'iptables'.

best ... khay
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Mon Feb 13, 2017 9:48 pm    Post subject: Reply with quote

ipkungfu has its own init too (/etc/init.d/ipkungfu), similarly deficient.

I have an /etc/init.d/iptables, but darned if I know where it came from. I may have created it as a copy of /etc/init.d/iptables, but if not, I've commandeered it, and mine is now a homebrew with a few extra commands.

Edit to note a typo -- what I have "commandeered" is an /etc/init.d/firewall


Last edited by cboldt on Mon Feb 13, 2017 11:06 pm; edited 1 time in total
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Mon Feb 13, 2017 10:26 pm    Post subject: Reply with quote

cboldt wrote:
ipkungfu has its own init too (/etc/init.d/ipkungfu), similarly deficient.

cboldt ... it isn't as prevelant as I'd initially thought:

ag -l 'provide firewall' ~portdir/net-firewall/**/files/*:
/var/pkg/gentoo/net-firewall/firehol/files/firehol.initrd
/var/pkg/gentoo/net-firewall/sanewall/files/sanewall.initd
/var/pkg/gentoo/net-firewall/shorewall/files/shorewall-lite.initd-r2
/var/pkg/gentoo/net-firewall/shorewall/files/shorewall.initd-r2
/var/pkg/gentoo/net-firewall/shorewall/files/shorewall6-lite.initd-r1
/var/pkg/gentoo/net-firewall/shorewall/files/shorewall6.initd-r1
/var/pkg/gentoo/net-firewall/ufw/files/ufw-2.initd

best ... khay
Back to top
View user's profile Send private message
jhon987
Apprentice
Apprentice


Joined: 18 Nov 2013
Posts: 225

PostPosted: Tue Feb 14, 2017 8:29 am    Post subject: Reply with quote

Thanks for the input guys, it seems that ipset fails to stop no matter its place in the shut down sequence.

I have iptables installed and used as a service (as far as I understand), if I recall correctly, when I installed ipset I remember I read that ipset is useless without iptables.
Anyhow so, I have iptables rule that reference ipset (-A INPUT -m set --match-set NAME src -j DROP).
while ipset is configured to start at 'boot' runlevel, iptables is at 'default' runlevel and both start without errors, therefore I gather that closing them in the opposite order should also work fine.
However, that's not the case (or so it seems, maybe I'm doing something wrong)...

I tried it before iptables, I tried after iptables, I also tried in between saving iptables and the stopping of it but every time I get the error message.


BEFORE
Code:
ipset            | * ipset is in use, can't stop
ipset            | * ERROR: ipset failed to stop
 [ ok ]
iptables         | * Saving iptables state ...
 [ ok ]
netmount         | * Unmounting network filesystems ...
consolekit       | * Stopping consolekit ...
swap             | * Deactivating swap devices ...
cronie           | * Stopping cronie ...
 [ ok ]
 [ ok ]
 [ ok ]
 [ ok ]
iptables         | * Stopping firewall ...
 [ ok ]


AFTER
Code:
iptables         | * Saving iptables state ...
local            | * Stopping local ...
urandom          | * Saving random seed ...
netmount         | * Unmounting network filesystems ...
cronie           | * Stopping cronie ...
ntpd             | * Stopping ntpd ...
 [ ok ]
swap             | * Deactivating swap devices ...
iptables         | * Stopping firewall ...
 [ ok ]
consolekit       | * Stopping consolekit ...
 [ ok ]
 [ ok ]
 [ ok ]
xdm              | * Stopping sddm ...
 [ ok ]
 [ ok ]
 [ ok ]
 [ ok ]
ipset            | * ipset is in use, can't stop
ipset            | * ERROR: ipset failed to stop


BETWEEN
Code:
iptables         | * Saving iptables state ...
ntpd             | * Stopping ntpd ...
consolekit       | * Stopping consolekit ...
ipset            | * ipset is in use, can't stop
ipset            | * ERROR: ipset failed to stop
cronie           | * Stopping cronie ...
 [ ok ]
swap             | * Deactivating swap devices ...
 [ ok ]
iptables         | * Stopping firewall ...
 [ ok ]


Edit: I did some experiments and yeah, stopping ipset after iptables works I think the issue is that openrc doesn't close ipset after iptables even though I've added: rc_ipset_after="iptables" in /etc/rc.conf
I'll see how I can solve that
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Tue Feb 14, 2017 1:14 pm    Post subject: Reply with quote

Or just do a homebrew /etc/init.d script that preserves the order. Pieces of mine (dearth of error checking, like missing saved iptables files and so forth). Variables from /etc/conf.d/firewall are brought in to /etc/init.d/firewall , and don't need to be "transferred" or reassigned to the lowercase values I've adopted.

The script DOES refer to the correct sources at start and stop, for example, use of ipsets at STOP refers to the active iptables arrangement. So, if you add ipsets during live configuration of the firewall, that will be saved correctly when the firewall shuts down.

Pardon the inconsistent indenting, I edited here after cut and paste, to get rid of some "overcomplicated for the suggestion" material in START.

Code:
# /etc/init.d/firewall - a homebrew

# Transfer assignments from /etc/conf.d/firewall
# Transfer assignments from /etc/conf.d/ipset

firewall_build=${IPTABLES_BUILD:=/usr/local/sbin/fw-build}
firewall_saved=${IPTABLES_SAVE:=/var/lib/iptables/iptables.rules}
ipset_saved=${IPSET_SAVE:=/var/lib/ipset/ipset.rules}
ipset_bin=${IPSET:=/usr/sbin/ipset}
iptables_bin=${IPTABLES:=/sbin/iptables}

start() {

using_ipsets=`grep -e "-m set" ${firewall_saved}`
if [ -n "$using_ipsets" ]; then
    ebegin "Loading saved kernel IP sets"
     $ipset_bin restore -exist < ${ipset_saved}
     eend $?
fi

ebegin "Restoring saved ${SVCNAME}"
${iptables_bin}-restore -c ${firewall_saved}
eend $?
}

stop() {
        using_ipsets=`${iptables_bin}-save | grep -e "-m set"`
        if [ -n "$using_ipsets" ]; then
                ebegin "Saving kernel IP sets"
                $ipset_bin save > ${ipset_saved}
                eend $?
        fi
        ebegin "Saving and Flushing ${SVCNAME}"
        ${iptables_bin}-save -c > ${firewall_saved}
        ${iptables_bin} --flush
        ${iptables_bin} --delete-chain
        eend $?
        if [ -n "$using_ipsets" ]; then
                ebegin "Flushing kernel IP sets"
                $ipset_bin flush
                $ipset_bin destroy
                eend $?
        fi
}


Not to say that openrc can't do the job, but I had other reasons for a homebrew firewall init, namely that the firewall changes between being on LAN and being away from home.

Edit to correct the location of "close-code" tag.
2nd edit to remark, some variables were transferred from /etc/conf.d/iptables in the first place. I edited /etc/conf.d/firewall by hand (meaning I probably created it from scratch), and if you don't have an /etc/conf.d/firewall, you could ...

Code:
cat /etc/conf.d/iptables /etc/conf.d/ipset >> /etc/conf.d/firewall


... then just use those variables directly, e.g., use ${IPTABLES_SAVE} instead of $firewall_saved, and skip the mess of variable reassignments at the top of the script.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Tue Feb 14, 2017 2:23 pm    Post subject: Reply with quote

jhon987 wrote:
Edit: I did some experiments and yeah, stopping ipset after iptables works I think the issue is that openrc doesn't close ipset after iptables even though I've added: rc_ipset_after="iptables" in /etc/rc.conf

jhon987 ... clear and unambigious explanations help us help you. Did you try 'rc_ipset_before="iptables"'? That should be all that's needed.

Note that as you have 'after' in rc.conf, and ipset is in the 'boot' runlevel, it will start 'iptables' in 'boot' as 'ipset is after iptables' (or in other words: 'iptables is before ipset')

Also your '-A INPUT -m set --match-set NAME src -j DROP', is this literal? It does match the 'NAME' used for '--create'? What does the following show:

Code:
# ipset --list | grep '^Name'

HTH & best ... khay
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Tue Feb 14, 2017 2:59 pm    Post subject: Reply with quote

Very good point, both services should be in the same runlevel, and I think "default" is the right place for that.

And after sharing my homebrew script, I cleaned it up, simplified slightly, and I see no reason that openrc couldn't handle this correctly. In my STOP routine, one could be misled into thinking that closing ipset necessarily has action both before after closing iptables. This isn't so. I created that misimpression by checking for ipsets in the running iptables, but it is just as valid to check for ipsets in the newly saved $IPTABLES_SAVE.

Which is what the script does now.

Code:
stop() {
        ebegin "Saving and flushing ${SVCNAME}"
        ${IPTABLES}-save -c > ${IPTABLES_SAVE}
        ${IPTABLES} --flush
        ${IPTABLES} --delete-chain
        eend $?

        if grep -q -e "-m set" ${IPTABLES_SAVE}; then
                ebegin "Saving and flushing kernel IP sets"
                ${IPSET} save > ${IPSET_SAVE}
                ${IPSET} flush
                ${IPSET} destroy
                eend $?
        fi
}
Back to top
View user's profile Send private message
jhon987
Apprentice
Apprentice


Joined: 18 Nov 2013
Posts: 225

PostPosted: Tue Feb 14, 2017 4:10 pm    Post subject: Reply with quote

Once again guys - thank you for the input!

cboldt < I really appreciate the time and effort you've invested in sharing your homebrew, however, personally, I rather use the solution with the least lines of code as possible.
I'm sure other people who'll encounter this thread in the future will put it to good use though - thanks.

khayyam < You're right I need to be clear as possible (I tend to forget that things that are clear in my mind aren't necessarily clear in others')
Yes, I did tried: 'rc_ipset_before="iptables"' I tried all kinds of combinations such as: 'rc_ipset_after="iptables"' and 'rc_iptables_before="ipset"' and so on... checking the log afterwards always showed that the error persisted.

When I issue 'rc-service ipset stop' in my terminal I get the same errors: * ipset is in use, can't stop | * ipset failed to stop.
However if I issue: 'rc-service iptables stop' and only then 'rc-service ipset stop' (in my terminal) then it stops without errors <- that's what I meant when I said "I did some experiments ..."

Maybe the term "after" doesn't mean that service-A is closed only after service-B has been terminated, maybe it merely means that service-A should initiate termination process after services-B initiated his (not necessarily after service-B already finished terminating). Perhaps it's a bug in opeRC, I don't know...

Anyways, I solved it! thanks to a nice fellow named timeBandit from this thread: https://forums.gentoo.org/viewtopic-t-778598-start-0.html
who said:
Quote:
When you stop a service, any services that need it will be stopped first.


So, for reference sake, the solution I used is: ' rc_iptables_need="ipset" '

P.S. khayyam you mentioned that you had this issue too and you solved it through: ' rc_ipset_before="firewall" ' <- it seem to me that ipset must be closed after the firewall (if the firewall is using ipset), so you might want to check your rc.log to make sure the error really is gone

Thank you both, you've directed me in the right direction.
PEACE
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Tue Feb 14, 2017 4:29 pm    Post subject: Reply with quote

rc_ipset_before="firewall" is a reference to the startup sequence. The stop sequence would be reversed from that.

Edit to add - the only value my homebrew has in this context is to show the relationship between iptables and ipset, outside of openrc. Plus, by sharing it, I looked at it more carefully and made improvements.

Whatever one is using for a firewall and ipsets can work together, by using the rc_ipset_before="firewall" setting.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Tue Feb 14, 2017 7:58 pm    Post subject: Reply with quote

jhon987 wrote:
khayyam < You're right I need to be clear as possible (I tend to forget that things that are clear in my mind aren't necessarily clear in others'). Yes, I did tried: 'rc_ipset_before="iptables"' I tried all kinds of combinations such as: 'rc_ipset_after="iptables"' and 'rc_iptables_before="ipset"' and so on... checking the log afterwards always showed that the error persisted.

jhon987 ... I think the issue here is you're changing the 'rc_ipset_before="iptables"' then {re-,}starting the specific service, so the previous order is in effect (rather than the order you would get from rc.conf). When testing you should be mindful of the (runlevel) dependencies (see: /run/openrc/deptree), rather than the specific service. Please try the following:

/etc/rc.conf:
rc_ipset_before="iptables"

Code:
# rc-update del ipset boot
# rc-update del iptables default
# rc-update del net.eth0 default
# rc-update del <other_network_related_service> default
# mkdir /etc/runlevels/online
# rc-update -s add default online # "stack" default runlevel onto online runlevel
# for i in net.eth0 ipset iptables <other_network_related_service> ; do rc-update add $i online ; done
# rc boot
# /etc/init.d/ipset status || /etc/init.d/ipset stop
# rc default
# rc online
# rc default

Note you can have the 'online' runlevel started at boot by providing 'softlevel=online' as a kernel parameter.

jhon987 wrote:
When I issue 'rc-service ipset stop' in my terminal I get the same errors: * ipset is in use, can't stop | * ipset failed to stop. However if I issue: 'rc-service iptables stop' and only then 'rc-service ipset stop' (in my terminal) then it stops without errors <- that's what I meant when I said "I did some experiments ..."

When you issue "rc-service ipset stop" the order is that of when the service was started (ie, in the 'boot' runlevel) this, I assume, is prior to any change to rc.conf. So, unless you change runlevel, or reboot, the changes in rc.conf won't have the desired effect.

jhon987 wrote:
Anyways, I solved it! thanks to a nice fellow named timeBandit from this thread: who said: "When you stop a service, any services that need it will be stopped first." So, for reference sake, the solution I used is: 'rc_iptables_need="ipset"'

This is the same as saying 'rc_ipset_before="iptables"'.

jhon987 wrote:
P.S. khayyam you mentioned that you had this issue too and you solved it through: ' rc_ipset_before="firewall" ' <- it seem to me that ipset must be closed after the firewall (if the firewall is using ipset), so you might want to check your rc.log to make sure the error really is gone

I assure you it works here, and the 'before' is the 'start' order, rc will do the reverse on 'stop'.

jhon987 wrote:
Thank you both, you've directed me in the right direction.

You're welcome & best ... khay
Back to top
View user's profile Send private message
jhon987
Apprentice
Apprentice


Joined: 18 Nov 2013
Posts: 225

PostPosted: Wed Feb 15, 2017 8:37 am    Post subject: Reply with quote

Quote:
cboldt: rc_ipset_before="firewall" is a reference to the startup sequence. The stop sequence would be reversed from that.

Oh, I assumed "before" means the service starts before and stops before <- thanks for clarifying that...

To both cboldt & khayyam < here's what I gather from my tests:
Since I have ipset in the boot runlevel and iptables in the default one thus they already behave as if there was a 'rc_ipset_before="iptables"'
and to add to that, I even added the 'rc_ipset_before="iptables"' to rc.conf as you can see in my second post on this thread which is comprised of sections from my rc.log I copied.

The fact of the matter is (and you can see that) no matter where the position of ipset was in the stop sequence it always popped up the same error.

Now, I have an assumption for why the before parameter doesn't work in my case: I use 'rc_parallel="Yes"' and furthermore I have 8-core Intel i7 processor.
Either one of these factors, or perhaps due to both at the same time, might have caused ipset to terminate even before iptables finished its termination process and despite the fact I used 'rc_ipset_before="iptables"'.
Since, from what I gather, the "before" statement is less strict than the "need" statement - I assume, the before was ineffective in my case whereas need is effective.

Using the "need" parameter has no backlash for me, if I remove ipset from the runlevels completely openRC still boots fine and no error is thrown whatsoever.

Best regards guys
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Wed Feb 15, 2017 11:14 am    Post subject: Reply with quote

I think the problem on your system is that the two services are in different runlevels!

The way I see it, the "before" and similar directives are meaningless when comparing two services that are on different runlevels. For example, if you ordered a service that is in the default runlevel to start before a service in the boot runlevel, your directive would (or should) be ignored. The services in "boot" are started before the services in "default."

Put both services ("ipset" and whatever you are using for a firewall, e.g. "iptables") in the "default" runlevel. They are meant to run "as a unit," and booting doesn't depend on ipset. Your network (other than "net.lo") is likely managed at the "default" runlevel too, and the network and the firewall are part of a bigger "unit."

Code:
rc-update del ipset boot
rc-update add ipset default
Back to top
View user's profile Send private message
cboldt
l33t
l33t


Joined: 24 Aug 2005
Posts: 829

PostPosted: Wed Feb 15, 2017 11:46 am    Post subject: Reply with quote

I see I ws wrong about inter-runlevel barriers. It appears there are none.

From /usr/share/doc/openrc-*/guide.md.bz2

Code:
# The Depend Function

This function declares the dependencies for a service script. This
determines the order the service scripts start.

```
depend() {
        need net
        use dns logger netmount
        want coolservice
}
```

`need` declares a hard dependency - net always needs to be started before this
        service does

`use` is a soft dependency - if dns, logger or netmount is in this runlevel
        start it before, but we don't care if it's not in this runlevel.
        `want` is between need and use - try to start coolservice if it is
        installed on the system, regardless of whether it is in the
        runlevel, but we don't care if it starts.

`before` declares that we need to be started before another service

`after` declares that we need to be started after another service, without
        creating a dependency (so on calling stop the two are independent)

`provide` allows multiple implementations to provide one service type, e.g.:
        `provide cron` is set in all cron-daemons, so any one of them started
        satisfies a cron dependency


I'd say "before" is not literally the reverse of "after." In the relationship between ipset and iptables, using "iptables after" allows at least the user to create an error by running `/etc/init.d/iptables stop` before stopping ipset.

The guide has more on the subject of service dependency.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum