iptables port forwarding on a router

[erg_samowzbudnik]

Hi all,

I want to ssh from internet to a server running behind home router running OpenWrt. Router has static IP. I'm a noob if it comes to iptables.

Assuming router address is 192.168.1.1 and server behind it 192.168.1.100, would adding those rules on the router be ok? Or am I supposed to use 'outside' IP rather than internal one for the router address?:

Code: Select all

iptables -t nat -A PREROUTING -p tcp --dport 22 -j DNAT -d 192.168.1.100:80
iptables -t nat -A POSTROUTING -p tcp -d 192.168.1.100 --dport 22 -j SNAT -s 192.168.1.1

Or should I rather use --to-destination and --to-source instead of -d and -s options?

[szatox]

Prerouting: if packet comes to router's public interface, change destination to your client
Postrouting: if packet comes to router's private interface, change source IP to your router's public IP
Not limiting by interfaces (or otherwise distinguishing between lan and wan traffic) will come at you and bite you at some point when you don't expect it. Also, it should be --sport in POSTROUTING in your case, since it's meant to match packets going from your sever in LAN to a client somewhere over the internet and you don't know his randomly chosen port at the time of creating those rules.

There is no need to alter "remote" end's IP for the client, because it will route via your router by default anyway.

Another thing you might need is an ACCEPT rule in the FORWARD chain.

[erg_samowzbudnik]

szatox,

thank you for that explanation, it slowly becomes more clear.

Only it would be only me connecting to said server using that port which is a standard ssh port, so I guess in this case I should skip --sport option, is that right?

Also, those eth0 - we are talking on interfaces on the server in pre and postrouting, is that correct?

So my rules would become

Code: Select all

iptables -t nat -A PREROUTING -p tcp --dport 22 -i eth0 -j DNAT -d 192.168.1.100:22
iptables -t nat -A POSTROUTING -p tcp -d 192.168.1.100 --dport 22 -o eth0 -j SNAT -s 92.xx.x.x

where 92.xx.x.x is the public facing IP, right?

[Hu]

I want to ssh from internet to a server running behind home router

Before you make this work, review the configuration of the interior sshd to ensure it is adequately hardened. Even with the alternate port you propose below, you are likely to get a large number of attack attempts. Hardening sshd now, before bots attack it, is less stressful than dealing with the attacks once they begin.

Code: Select all

iptables -t nat -A PREROUTING -p tcp --dport 22 -j DNAT -d 192.168.1.100:80

szatox already critiqued this one. I would add the interface qualifier like he suggested. The rule as written is incorrect relative to your stated intent. According to the documentation, that rule should not even parse correctly. You cannot use a :port qualifier with -d. -d is a matcher, not a rewriter, so you are saying that if the destination is already correct (which it should not be, unless someone on the Internet is doing something very odd and your ISP plays along), then engage the DNAT target and do no rewrites, because you didn't tell DNAT what to do. Incorporating the interface criticism, I would rewrite this as:

Code: Select all

iptables -t nat -A PREROUTING -i eth0 -p tcp -d 192.168.1.1 --dport 22 -j DNAT --to-destination 192.168.1.100:80

I like putting a destination qualifier on the rewrite rule as a secondary check that it not match odd traffic. This assumes that your public IP is sufficiently predictable that you can put it in the rule and not need to update it routinely. If you can't assume that, drop the -d 192.168.1.1 and leave the rest of the rule.

Code: Select all

iptables -t nat -A POSTROUTING -p tcp -d 192.168.1.100 --dport 22 -j SNAT -s 192.168.1.1

Remove this rule entirely. Its only purpose is to convince your sshd that all connections are initiated from the router, which is at best a lie that will reduce the value of your logs and at worst a security problem, since it prevents the sshd from applying any source-based restrictions. Assuming your internal systems have working Internet access as-is through some other POSTROUTING rule, there is no need for any POSTROUTING rule specific to this project.

[erg_samowzbudnik]

Hu,

sir, thank you for spending your time on this. I understand it must be frustrating looking at lame setup attempts like this one.

I'm still having troubles with this. Perhaps I explain my setup further.

First firewall is on the antenae (it's fairly remote, I get internet over the air) which is also a router - my ISP has access to this, I asked him to open some ports and port 22 looks open:

Code: Select all

nmap -Pn 92.xx.xx.xx
...
22/tcp   filtered ssh
...

Behind this seats my router with a firewall. LAN is on br-lan, WAN on eth0.1. I have added the rule as you perscribed. The -i option, that would be the LAN on the router, is that correct?

I use no custom port for ssh, it's 22.

As I understand, my rule would become:

Code: Select all

iptables -t nat -A PREROUTING -i br-lan -p tcp -d 192.168.1.1 --dport 22 -j DNAT --to-destination 192.168.1.100:22

Also in my router configuration panel I see a box 'Allow remote hosts to connect to local SSH forwarded ports ', should that be ticked also? I mean, I did toggle it but that doesn't allow me to ssh to the server behind the router either.

Yes, I do have static IP and shouldn't need to update it.
Again, thanks for looking into this.

[erg_samowzbudnik]

I realised dropbear is listening on port 22 on the router, changed that port, still no joy.

[Hu]

sir, thank you for spending your time on this. I understand it must be frustrating looking at lame setup attempts like this one.

Actually, it's rather nice. It's been ages since I got to critique someone's iptables setup.

my ISP has access to this, I asked him to open some ports and port 22 looks open:

Code: Select all

nmap -Pn 92.xx.xx.xx
...
22/tcp   filtered ssh
...

That does not look open to me. If it were open, you would get:

Code: Select all

PORT   STATE SERVICE
22/tcp open  ssh

If it is not open, you will definitely not get ssh connectivity from the system that ran nmap to the system that you are mapping. You need to fix this line. Now, if we're lucky, what happened here is that your ISP routed the traffic to you, then your iptables rules mishandled the traffic in a way that the connection was marked as filtered. If so, then we just need to fix your iptables rules. If not, and your ISP is filtering it (yes, you said they aren't, but I place no faith in their statement), then that is a problem because you would need to get them to stop filtering it. Let's start with the assumption that your ISP is cooperating and we need to fix your rules.

Behind this seats my router with a firewall. LAN is on br-lan, WAN on eth0.1. I have added the rule as you perscribed. The -i option, that would be the LAN on the router, is that correct?

-i matches the network interface on which the traffic arrived, as seen by the system evaluating the rule. You're trying to allow traffic from the Internet to an internal system, so -i needs to name the WAN card.

I use no custom port for ssh, it's 22.

As I understand, my rule would become:

Code: Select all

iptables -t nat -A PREROUTING -i br-lan -p tcp -d 192.168.1.1 --dport 22 -j DNAT --to-destination 192.168.1.100:22

I think you need either -i eth0 or maybe -i eth0.1 (why is your card .1?). Otherwise, that line looks good.

Also in my router configuration panel I see a box 'Allow remote hosts to connect to local SSH forwarded ports ', should that be ticked also? I mean, I did toggle it but that doesn't allow me to ssh to the server behind the router either.

I think that refers to GatewayPorts on the router itself. If so, then it is irrelevant here since you aren't trying to ssh to the router. You're trying to have the router pass a TCP stream (which happens to contain ssh traffic) to a different system.

[papas]

does your default forward chain, allow traffic to destination?

[erg_samowzbudnik]

Hu,

This

If not, and your ISP is filtering it (yes, you said they aren't, but I place no faith in their statement)

Lesson from here? Trust but verify.

Long horror story short, I run a long debugging session with my IPS last night and no, ports are not open. Turns out on OpenWrt setting up firewall rules should be done either through web interface or through config files (/etc/firewall.user or /etc/config/firewall). I did that. Set my IP to the subnet of the WAN interface (192.168.88.xx), killed NetworkManager, took down wireless card to be sure I'm not connecting through LAN, set static IP on the ethernet of the laptop, ssh to pi gets forwarded and I can log in. So I guess setup on my side works. Would that be correct though? When he opens that port on the antenna/router before my OpenWrt router, it should just work.com, right? Or am I missing something?

Now I plug the antenna cable back to the WAN port, ssh through LAN to the router, run tcpdump on the OpenWrt router on port 22 and wait. Get someone to try to ssh through public IP from the internet and I see no incoming packets. As I understand, tcpdump should catch those packets coming from the outside before they get dropped by the firewall. Would that be correct?

It took some time before I made my ISP to acknowledge that it's their fault ('but we have tons of setups like that and they all work, OpenWrt is known to be unreliable, nmap doesn't show stuff correctly' sort of stuff), he had to install (sic!) nmap on his windows machine (sic!) to see what I'm telling him (port 22 filtered) ...

So it's up to them to figure out what is wrong with the setup on the antenna/router. But, being a small business, they are not going to be able to do it right away. That router/antenna in question is Mikrotik lhg5 running software (?) version 6.44.6

He wouldn't give me access to it. I know, I bloody paid for that gear, but he is the one wearing pants here. Tough. But he did give me dump of the config files on it. It is a little scary to me as someone is running services on the thing and I have no way of hardening it. Some of those are visible in the configs, some are not (I found the port he uses to ssh to it which does not appear here, there's some other open ports running weird services ...)

So now the question becomes: is it possible to figure out what is wrong with those config files that they do not forward my ssh traffic?

Here they come:

/ip firewall filter> print Flags: X - disabled, I - invalid, D - dynamic 0 D ;;; special dummy rule to show fasttrack counters
chain=forward action=passthrough
1 ;;; defconf: accept ICMP
chain=input action=accept protocol=icmp log=no log-prefix=""
2 ;;; defconf: accept established,related
chain=input action=accept connection-state=established,related log=no
log-prefix=""
3 X ;;; defconf: drop all from WAN
chain=input action=drop in-interface=wlan1 log=no log-prefix=""
4 ;;; defconf: fasttrack
chain=forward action=fasttrack-connection
connection-state=established,related log=no log-prefix=""
5 ;;; defconf: accept established,related
chain=forward action=accept connection-state=established,related log=no
log-prefix=""
6 X ;;; defconf: drop invalid
chain=forward action=drop connection-state=invalid log=no log-prefix=""
7 ;;; defconf: drop all from WAN not DSTNATed
chain=forward action=drop connection-state=new
connection-nat-state=!dstnat in-interface=wlan1 log=no log-prefix=""



/ip firewall nat> print Flags: X - disabled, I - invalid, D - dynamic 0 ;;; defconf: masquerade
chain=srcnat action=masquerade out-interface=wlan1 log=no log-prefix=""
1 ;;; 2
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=333 protocol=tcp
dst-address=92.xx.xx.xx dst-port=333 log=no log-prefix=""
2 ;;; 2udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=333 protocol=udp
dst-address=92.xx.xx.xx dst-port=333 log=no log-prefix=""
3 ;;; 3
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=80 protocol=tcp
in-interface=wlan1 dst-port=80 log=yes log-prefix=""
4 ;;; 3udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=80 protocol=udp
in-interface=wlan1 dst-port=80 log=no log-prefix=""
5 ;;; 4
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=443 protocol=tcp
dst-address=92.xx.xx.xx dst-port=443 log=no log-prefix=""
6 ;;; 4
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=22 protocol=tcp
dst-address=92.xx.xx.xx dst-port=22 log=yes log-prefix=""
7 ;;; 4udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=443 protocol=udp
dst-address=92.xx.xx.xx dst-port=443 log=no log-prefix=""
8 ;;; 4udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=22 protocol=udp
dst-address=92.xx.xx.xx dst-port=22 log=yes log-prefix=""

[Hu]

Would that be correct though? When he opens that port on the antenna/router before my OpenWrt router, it should just work.com, right? Or am I missing something?

If I understood correctly what you did, your test was good, and shows that your configuration is correct.

Now I plug the antenna cable back to the WAN port, ssh through LAN to the router, run tcpdump on the OpenWrt router on port 22 and wait. Get someone to try to ssh through public IP from the internet and I see no incoming packets. As I understand, tcpdump should catch those packets coming from the outside before they get dropped by the firewall. Would that be correct?

Yes, I believe your understanding is correct. That definitely works when using a standard Linux, like Gentoo itself, as the router, and I expect OpenWrt would work the same way.

It took some time before I made my ISP to acknowledge that it's their fault ('but we have tons of setups like that and they all work, OpenWrt is known to be unreliable, nmap doesn't show stuff correctly' sort of stuff), he had to install (sic!) nmap on his windows machine (sic!) to see what I'm telling him (port 22 filtered) ...

I wish I could say I was surprised, but that is exactly the sort of story I was expecting.

So now the question becomes: is it possible to figure out what is wrong with those config files that they do not forward my ssh traffic?

Maybe. The command language is ugly, but I can guess at what it is telling us.

Here they come:

/ip firewall filter> print Flags: X - disabled, I - invalid, D - dynamic 0 D ;;; special dummy rule to show fasttrack counters
chain=forward action=passthrough
3 X ;;; defconf: drop all from WAN
chain=input action=drop in-interface=wlan1 log=no log-prefix=""

This one would prevent accessing the ISP router, but it is currently disabled. My guess is that disabling this is what they did when they said they configured it correctly.

/ip firewall nat> print Flags: X - disabled, I - invalid, D - dynamic 0 ;;; defconf: masquerade
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=80 protocol=tcp
in-interface=wlan1 dst-port=80 log=yes log-prefix=""

Do you want to host an http server? If not, this rule should be removed.

4 ;;; 3udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=80 protocol=udp
in-interface=wlan1 dst-port=80 log=no log-prefix=""

This rule looks useless and should be removed. HTTP is not normally done on UDP.

5 ;;; 4
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=443 protocol=tcp

Same comment for HTTPS hosting.

dst-address=92.xx.xx.xx dst-port=443 log=no log-prefix=""
6 ;;; 4
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=22 protocol=tcp
dst-address=92.xx.xx.xx dst-port=22 log=yes log-prefix=""

This one looks valuable. Is that the right to-address for the next stage router?

7 ;;; 4udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=443 protocol=udp
dst-address=92.xx.xx.xx dst-port=443 log=no log-prefix=""

This is useless. HTTPS is not usually done over UDP.

8 ;;; 4udp
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=22 protocol=udp
dst-address=92.xx.xx.xx dst-port=22 log=yes log-prefix=""

This is useless. ssh is not done over UDP.

[erg_samowzbudnik]

Hu,

6 ;;; 4
chain=dstnat action=dst-nat to-addresses=192.168.88.254 to-ports=22 protocol=tcp
dst-address=92.xx.xx.xx dst-port=22 log=yes log-prefix=""

Yes, that is the correct address. Apart from cruft (should I consider those open UDP ports a vulnerability?) this is the only thing that caught my eye as well.

But the problem is, this rule looks exactly like the one for ports 80 or 443 and packets are arriving on those ports. So my thinking goes, there's some superseding setting somewhere in the guts of that router/antenna we're not seeing that is keeping port 22 as filtered. Or is it possible that, even though the rule is there and syntax is ok, that port is being used for something else by the antenna itself and that is why forwarding rule doesn't do it's job?

And yes, I do intend to run a http(s) server so those are needed.

[erg_samowzbudnik]

What I am trying to do now is, since traffic seems to be forwarded through port 333, to allow ssh forwarding through that port.

But something is stopping me. I have changed default port in both /etc/ssh/ssh_config and /etc/ssh/sshd_config I allowed any address in those as well but connection is timing out. I still have ssh connections running to the Pi, restarted sshd but connection is still timing out. Am I missing something?

[erg_samowzbudnik]

Gosh. Got there in the end.

Forwarded traffic from port 333 on the router to port 22 on Pi and - blam! - it works.

For some curious reason adding rules via /etc/firewall.user on OpenWrt didn't do it but adding it through routers web interface did.

Should I mark it as solved? I mean, it is a workaround, that bloody port 22 on the antenna still doesn't do what it's supposed to do but the workaround works.

Hu,

thank you for staying with me on this one. Many thanks. Amazing.

[Hu]

Yes, that is the correct address.

That is disappointing. That was my only lead for what would be wrong with it, so I am happy to see your follow-up post that you found a solution of sorts.

Apart from cruft (should I consider those open UDP ports a vulnerability?)

Technically, maybe a small denial-of-service risk, since you will be dropping those on your OpenWrt if they make it that far. Dropping them on the ISP router would save the bandwidth/CPU cost of sending them to you. Realistically, the link between ISP and OpenWrt is probably not your bottleneck, so dropping unwanted traffic at the ISP layer won't help you much, if at all, if you come under a flood of unsolicited UDP traffic.

But the problem is, this rule looks exactly like the one for ports 80 or 443 and packets are arriving on those ports. So my thinking goes, there's some superseding setting somewhere in the guts of that router/antenna we're not seeing that is keeping port 22 as filtered. Or is it possible that, even though the rule is there and syntax is ok, that port is being used for something else by the antenna itself and that is why forwarding rule doesn't do it's job?

All those are reasonable possibilities that I cannot rule out from the information available. I am curious to know what IP/port pair your ISP's technician uses to access the device, since that could fit into your theory that the port is busy elsewhere. Hypothetically, the device could have a rule that directs traffic from the ISP's Windows system to the local sshd process, and drops other traffic. I would have expected to see that in the rules you posted though.

And yes, I do intend to run a http(s) server so those are needed.

If that's intended, then that is fine.

Forwarded traffic from port 333 on the router to port 22 on Pi and - blam! - it works.

Nicely done.

For some curious reason adding rules via /etc/firewall.user on OpenWrt didn't do it but adding it through routers web interface did.

The file may not be watched for modification, so you might need to restart the firewall script, or even the entire system, to make those changes live. If you are unlucky, that file might also be where OpenWrt saves the rules it has loaded, in which case an orderly shutdown might overwrite your changes.

Should I mark it as solved? I mean, it is a workaround, that bloody port 22 on the antenna still doesn't do what it's supposed to do but the workaround works.

Up to you. Without the ability to do more analysis directly on the antenna, I don't have any suggestions for how to pursue this further. We know:

• The device can forward TCP traffic on port 333.
• The device claims it forwards TCP traffic on port 22.
• TCP/22 traffic does not work end-to-end.

We think we know:

• If TCP/22 made it from the antenna to OpenWrt, then OpenWrt would do the right thing.

You could use a packet capture done on the OpenWrt to confirm that TCP/22 from the Internet, addressed to the antenna's public address, does not result in any observable traffic on the WAN side of the OpenWrt. If no traffic is observed, then we would need to start collecting packet captures on the antenna, both upstream and downstream interfaces - which as you stated above, you don't have permission to do. Whether your ISP would cooperate on that (there is no security reason not to, but they may not want to bother), and whether you want to spend the time trying to get them to do it, is not something I can answer. Alternatively, if the packet capture on the OpenWrt shows that the antenna is forwarding you port 22 traffic, then we need to revisit the OpenWrt to find why the traffic got lost. If you want to pursue this further, I'm happy to continue providing what advice I can.

As a funny aside, being forced to use an unusual port may have some minor security benefit to you. Botnets routinely scan for an open port 22 and attack it. Some botnets scan for ssh on other ports, but the attack volume seems to be lower. For this purpose, the public IP/port pair is what counts, so the bots would need to attack antenna/333, even though internally you have the port mapped back to 22 on the Pi.