| View previous topic :: View next topic |
| Author |
Message |
Beju
Apprentice
Joined: 30 Mar 2006
Posts: 171
|
 Posted: Mon Nov 15, 2010 8:59 pm Post subject: iptables very slow after upgrade |
 |
|
Hello,
I was using iptables 1.4.3.2 as my firewall. Some time ago, a did an emerge -Dupv world, and iptables was upgraded to 1.4.6. Now, when I execute my firewall script, it is four or five times slower than with 1.4.3.2 version. I tried some higher versions also, like 1.4.9, but they run EVEN slower than that.
Was something changed in the iptables recently to make it work so slow? Or maybe it's some kind of package conflict (didn't touch my kernel during upgrade).
Thanks in advance. |
|
| Back to top |
|
 |
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21633
|
| Posted: Tue Nov 16, 2010 12:15 am Post subject: |
|
|
| What iptables script do you mean? /etc/init.d/iptables should be quite fast. |
|
| Back to top |
|
|
Beju
Apprentice
Joined: 30 Mar 2006
Posts: 171
|
| Posted: Tue Nov 16, 2010 12:26 am Post subject: |
|
|
I mean my custom firewall script  It just executes a bunch of iptables rules. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21633
|
| Posted: Tue Nov 16, 2010 4:26 am Post subject: |
|
|
| That is what I was afraid you would say. Why are you using a custom script? Can you not express your rules through the standard Gentoo iptables initscript? |
|
| Back to top |
|
|
Beju
Apprentice
Joined: 30 Mar 2006
Posts: 171
|
| Posted: Thu Nov 18, 2010 11:27 pm Post subject: |
|
|
AFAIK, gentoo iptables initscript (/etc/init.d/iptables) is not responsible for loading any iptables rules. It just executes iptables-save or iptables-restore and does a lot of kernel checks.
Because my firewall is rather complicated, I put all the rules and logic in a bash script, and this is tha script that suddenly became slow.
Going a little ahead:
- I've checked it and I'm sure that the package responsible for the slowdown is the iptables itself. Not kernel, not some missing USE flag, etc..
- Because of the the firewall specifics, the gentoo initscript and the whole iptables-save / -restore play is useless for me. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21633
|
| Posted: Fri Nov 19, 2010 3:44 am Post subject: |
|
|
| Beju wrote: | | AFAIK, gentoo iptables initscript (/etc/init.d/iptables) is not responsible for loading any iptables rules. It just executes iptables-save or iptables-restore and does a lot of kernel checks. |
Running iptables-restore does load the iptables rules.
| Beju wrote: | Going a little ahead:
- I've checked it and I'm sure that the package responsible for the slowdown is the iptables itself. Not kernel, not some missing USE flag, etc..
- Because of the the firewall specifics, the gentoo initscript and the whole iptables-save / -restore play is useless for me. |
Without seeing the script, there is no way for us to help further, nor can I offer any guidance on how you could make better use of the atomic restoration functionality. |
|
| Back to top |
|
|
Beju
Apprentice
Joined: 30 Mar 2006
Posts: 171
|
| Posted: Fri Nov 19, 2010 7:38 pm Post subject: |
|
|
| Hu wrote: | | Running iptables-restore does load the iptables rules. |
Yes, that's true, but it has to load them from somewhere, right? So, it means I have to save them before I can use the iptables-restore. Before I can save them, I have to execute them in the "classic" way (either manually or using a script).
Now goes the funny thing:
Because I'm reloading the rules only when they change, iptables-restore is useless for me, since to use it I have to generate the rules using my script first, which basically takes me back to the main problem.
| Hu wrote: | | Without seeing the script, there is no way for us to help further, nor can I offer any guidance on how you could make better use of the atomic restoration functionality. |
Don't get me wrong, I'm not asking how to use the iptables-restore by any means possible instead of the "classic" way of adding rules, because mainly I have to change them, not restore them.
I don't know why do you want to see the script. Just imagine about a thousand lines long bash script with iptables rules inside it. These rules are generated dynamically and depend on bunch of parameters, that's way saving them once and restoring them all the time is not an option. My question is simple:
Why adding rules with iptables-1.4.6 is few times slower than with iptables-1.4.3.2?
And by "iptables", I don't mean any gentoo, bash or any other script, I mean the iptables executable. |
|
| Back to top |
|
|
chiefbag
Guru
Joined: 01 Oct 2010
Posts: 542
Location: The Kingdom
|
| Posted: Fri Nov 19, 2010 11:10 pm Post subject: |
|
|
Curious?
You are correct about iptables-save will only output the current rules to tty if these need to be saved then they need > rules-save.
You are also correct that it is not possible to get tricky configs by adding an iptable rule from the command as stuff like SNAT etc cannot be accomplished through this method. I know this is not of much use to you as you have already outlined this afore.
I wonder if this issue may be related to a kernel upgrade. ie a doddgy module that may be required or incompatible with that version of ip tables.
I would try loading iptables with no rules-save file and see if it is still as slow.
If so check dmesg and look for any errors. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21633
|
| Posted: Sat Nov 20, 2010 3:30 am Post subject: |
|
|
| Beju wrote: | | Yes, that's true, but it has to load them from somewhere, right? |
It loads them from stdin. There is no requirement that what it reads from stdin was generated by iptables-save. If you can generate well formed text in the layout that iptables-restore expects, you can generate rules and feed them to it. I do this on some systems with especially complex rules that cannot be stored statically.
| Beju wrote: | | I don't know why do you want to see the script. Just imagine about a thousand lines long bash script with iptables rules inside it. |
I wanted to see the script because I expected to see some construct, such as reliance on symbolic lookups, which might have changed behavior. If it is that long, then I do not want to see it. Also, I would strongly suggest rewriting that script in a more maintainable language. Shell script is good for quick jobs, but once it gets that big, you are often better off using a language which has more native functionality.
| Beju wrote: | | Why adding rules with iptables-1.4.6 is few times slower than with iptables-1.4.3.2? |
I have no idea. Without knowing what commands you are executing and the distribution of the slowdown, I cannot even begin to speculate. Are all the executed commands equally slow or are there some commands which are slower than others? Is the total slowdown a function of the sheer number of rules generated or are certain operations especially slow?
| chiefbag wrote: | | You are also correct that it is not possible to get tricky configs by adding an iptable rule from the command as stuff like SNAT etc cannot be accomplished through this method. |
Could you elaborate on this? Both SNAT and DNAT can be added through the iptables command line. To my knowledge, any individual rule that can be loaded into iptables can be expressed via running /sbin/iptables with the proper arguments. |
|
| Back to top |
|
|
Beju
Apprentice
Joined: 30 Mar 2006
Posts: 171
|
| Posted: Sat Nov 20, 2010 2:44 pm Post subject: |
|
|
| Hu wrote: | | It loads them from stdin. There is no requirement that what it reads from stdin was generated by iptables-save. If you can generate well formed text in the layout that iptables-restore expects, you can generate rules and feed them to it. I do this on some systems with especially complex rules that cannot be stored statically. |
I was afraid that you'd suggest to generate the rules in format compatible with iptables-restore  I also thought about that, but it seems just too complicated to accomplish with bash (or maybe I am wrong?). Also, will the overhead caused by this "generator" not be bigger than the actual slowdown I'm experiencing now?
I was wondering, if the "iptables-restore" way is so efficient, a proper tool would already exist to generate the format needed by iptables-restore directly, bypassing the kernel calls.
| Hu wrote: | | Also, I would strongly suggest rewriting that script in a more maintainable language. Shell script is good for quick jobs, but once it gets that big, you are often better off using a language which has more native functionality. |
I think it's a good idea:) What language would you suggest?
| Hu wrote: | | Are all the executed commands equally slow or are there some commands which are slower than others? Is the total slowdown a function of the sheer number of rules generated or are certain operations especially slow? |
I think the main problem is with rule addition (iptables -A). If you execute it once, the difference is negligible, but if you execute it hundreds of times... |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21633
|
| Posted: Sat Nov 20, 2010 9:55 pm Post subject: |
|
|
| Beju wrote: | | I was afraid that you'd suggest to generate the rules in format compatible with iptables-restore I also thought about that, but it seems just too complicated to accomplish with bash (or maybe I am wrong?). |
I used Perl for mine. You could also use Python if you are more comfortable with that. Bash can do it, but I find that large programs are often more efficient if done in a language that does not require frequent use of child processes for basic tasks.
| Beju wrote: | | Also, will the overhead caused by this "generator" not be bigger than the actual slowdown I'm experiencing now? |
Maybe. Since we do not know why you are seeing slowness now, it is hard to say whether this would be better or worse.
| Beju wrote: | | I was wondering, if the "iptables-restore" way is so efficient, a proper tool would already exist to generate the format needed by iptables-restore directly, bypassing the kernel calls. |
You mean like iptables-save? Using /sbin/iptables is great for minor tweaks to the system, whereas iptables-save / iptables-restore are good for bulk operations.
| Beju wrote: | | I think the main problem is with rule addition (iptables -A). If you execute it once, the difference is negligible, but if you execute it hundreds of times... |
Yes, I expected rule addition would dominate. However, I was more interested in the type of rules you are adding. For example, do you use hostnames that iptables must resolve before it can add the rule? Do you use a large number of extended match rules via the -m option to iptables? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
