OpenVPN, don't route outbound traffic through VPN?

[Akaihiryuu]

I'm using a VPN so I can get a public IP address for my inbound traffic (my internet service does not give me a public IP). The public IP is taken care of. Inbound traffic works fine. The problem is, my OUTBOUND traffic is also being routed through the VPN, and I don't want this to happen. I want all traffic originating on my network to go out through the regular internet connection. I want the VPN to be used for inbound only. I don't know enough about OpenVPN to have any clue on how to do this, and the documentation is really inadequate. Googling doesn't help either, as it seems most people using it want to route all traffic through the VPN and not what I'm doing. Anyway, here is my current seutp:

OpenVPN config:

Code: Select all

client
dev tun
proto tcp
remote vleu-usoh1-ovpn-tcp.pointtoserver.com 80
persist-key
persist-tun
ca ca.crt
tls-auth Wdc.key 1
cipher AES-256-CBC
comp-lzo
verb 1
mute 20
redirect-private
route 192.168.101.0/24
#route-method exe
#route-delay 2
#route 0.0.0.0 0.0.0.0
float
auth-user-pass pass.txt
auth-retry interact
ifconfig-nowarn

Current routing table:

Code: Select all

triforce openvpn # route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         178.170.139.193 128.0.0.0       UG    0      0        0 tun0
default         192.168.101.1   0.0.0.0         UG    5      0        0 eth1
loopback        localhost       255.0.0.0       UG    0      0        0 lo
128.0.0.0       178.170.139.193 128.0.0.0       UG    0      0        0 tun0
178.170.139.2   192.168.101.1   255.255.255.255 UGH   0      0        0 eth1
178.170.139.192 0.0.0.0         255.255.255.192 U     0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
192.168.101.0   0.0.0.0         255.255.255.0   U     5      0        0 eth1

What do I need to do? So far I've been fiddling with various options in my OpenVPN config file but nothing I do seems to stop it from setting that 0.0.0.0 route to tun0. I want my outbound traffic going through eth1, not tun0.

[chiefbag]

Is this question not a rehash of your original post? https://forums.gentoo.org/viewtopic-t-1068420.html

In any case adding the following to your client config will avoid the default route from being changed and all traffic routed out tun0.

Code: Select all

route-nopull

--EDIT--

From the OpenVPN docs

https://community.openvpn.net/openvpn/w ... n23ManPage[/code]

--route-nopull
When used with --client or --pull, accept options pushed by server EXCEPT for routes, block-outside-dns and dhcp options like DNS servers.
When used on the client, this option effectively bars the server from adding routes to the client's routing table, however note that this option still allows the server to set the TCP/IP properties of the client's TUN/TAP interface.

[Akaihiryuu]

Is this question not a rehash of your original post? https://forums.gentoo.org/viewtopic-t-1068420.html

In any case adding the following to your client config will avoid the default route from being changed and all traffic routed out tun0.

Code: Select all

route-nopull

--EDIT--

From the OpenVPN docs

https://community.openvpn.net/openvpn/w ... n23ManPage[/code]

--route-nopull
When used with --client or --pull, accept options pushed by server EXCEPT for routes, block-outside-dns and dhcp options like DNS servers.
When used on the client, this option effectively bars the server from adding routes to the client's routing table, however note that this option still allows the server to set the TCP/IP properties of the client's TUN/TAP interface.

Ok, that sort of worked. My outbound traffic is no longer routing through the VPN, but my inbound connections aren't working properly. I have a feeling I have to add another route to make it work but I have no clue what I'm doing at this point. Actually, it would be more accurate to say it sort of works, but really really slow. I can't believe this is so difficult, you'd think this type of configuration would be extremely common.

[Akaihiryuu]

Ok, I've decided to try to go about this a different way. Rather than only trying to do inbound via the VPN and mess with routing tables, I've decided to just concentrate on the ports that I need to be able to connect to from outside. That will be, 8022, 8080, 8443, 8888, and 8889. Is it possible to use route-nopull, and then use iptables to only route these (source) ports through the VPN, while just running everything else through my regular connection? If this works I can just add an iptables line to my NAT table and just use -m multiport --sports to match all the ports I want going through the VPN. And if I'm correct, I would do these via PREROUTING in the NAT table?

[chiefbag]

If the services you want to allow access to are running/bound to the tun0 interface then you can use an INPUT rule.
Otherwise you could use PREROUTING or FORWARD.
Don't forget to SNAT or MASQUERADE traffic -o tun0 and allow Related established on the INPUT or FORWARD, PREROUTING

[Akaihiryuu]

Here's what I've got so far. I don't want to take the server down to implement it yet because my roommate runs a muck and a couple other things that I don't want to take down unless I know what I'm doing is going to work and it will only be down for a minute. The idea here is that I will put route-nopull in openvpn, so that the default route is my regular internet connection. And then snat/masquerade only the ports I want to tun0 (which are the ones in the first two lines in postrouting - using source ports). Until I make the openvpn change, my default routes are going through tun0, so even if I try to masquerade stuff out eth1 it doesn't work (it goes out tun0 anyway). Does this look good before I try it? Is it really as simple as just using snat/masquerade to direct certain stuff to a different interface? I don't generally bind services to specific interfaces, with the exception of certain things I ONLY want available in the LAN and don't want to be accessible from outside. Anything I want accessible from outside should be bound to all interfaces. This is actually almost identical to my regular iptables setup from back when I actually had an IP address without having to use a VPN...the only tinkering I did was in the postrouting chain.

Code: Select all

Chain INPUT (policy DROP 7 packets, 2037 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  any    any     localhost            anywhere
    3   603 ACCEPT     all  --  any    any     192.168.0.0/24       anywhere
   57  7668 ACCEPT     all  --  any    any     anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere             multiport dports 8022,http-alt,8443,64738
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere             multiport dports 1069,3000,8888,8889,git

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  any    any     192.168.0.0/24       anywhere
    0     0 ACCEPT     all  --  any    any     anywhere             192.168.0.0/24

Chain OUTPUT (policy ACCEPT 48 packets, 6565 bytes)
 pkts bytes target     prot opt in     out     source               destination

Code: Select all

Chain PREROUTING (policy ACCEPT 119 packets, 34627 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 7 packets, 661 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 7 packets, 661 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  tcp  --  any    tun0    anywhere             anywhere             multiport sports 8022,http-alt,8443
    0     0 MASQUERADE  tcp  --  any    tun0    anywhere             anywhere             multiport sports 1069,3000,8888,8889,git
    0     0 MASQUERADE  all  --  any    eth1    192.168.0.0/24       anywhere

[chiefbag]

Just restart your OpenVPN client with the no route option, it will not cause any major outage.
As for iptables you should be able to insert and delete rules on the fly which can make testing easier.
Also if asking for help with rules post the actual rules rather then the -L command as it is much easier to read and comment on a rule.

[Akaihiryuu]

Just restart your OpenVPN client with the no route option, it will not cause any major outage.
As for iptables you should be able to insert and delete rules on the fly which can make testing easier.
Also if asking for help with rules post the actual rules rather then the -L command as it is much easier to read and comment on a rule.

Actually, I tried restarting OpenVPN with route-nopull and my inbound connections stopped working completely. That's why I want to make sure before I do it again.

Anyway, for the most part I'm doing my iptables stuff in a script rather than typing the rules out manually, so here it is:

Code: Select all

IPT="/sbin/iptables"
NAT="/sbin/iptables -t nat"

${IPT} -P INPUT ACCEPT
${IPT} -P FORWARD ACCEPT
${IPT} -F INPUT
${IPT} -F FORWARD
${NAT} -F PREROUTING
${NAT} -F POSTROUTING

if [ "$1" = "stop" ]
then
        exit
fi

${IPT} -P INPUT DROP
${IPT} -A INPUT -s 127.0.0.1 -j ACCEPT
${IPT} -A INPUT -s 192.168.0.0/24 -j ACCEPT
${IPT} -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
${IPT} -A INPUT -p tcp -m multiport --dports 8022,8080,8443,64738 -j ACCEPT
${IPT} -A INPUT -p tcp -m multiport --dports 1069,3000,8888,8889,9418 -j ACCEPT

${IPT} -P FORWARD DROP
${IPT} -A FORWARD -s 192.168.0.0/24 -j ACCEPT
${IPT} -A FORWARD -d 192.168.0.0/24 -j ACCEPT

${NAT} -A POSTROUTING -p tcp -m multiport --sports 8022,8080,8443 -o tun0 -j MASQUERADE
${NAT} -A POSTROUTING -p tcp -m multiport --sports 1069,3000,8888,8889,9418 -o tun0 -j MASQUERADE
${NAT} -A POSTROUTING -o eth1 -s 192.168.0.0/24 -j MASQUERADE

[Hu]

When chiefbag asked for the rules, I believe he was expecting you to post the output of iptables-save, which is also the format you should be using to load your rules. Loading via a script is error-prone and racy.

Code: Select all

${IPT} -P INPUT ACCEPT
${IPT} -P FORWARD ACCEPT

You should not change the policy to accept if you plan to change it to drop later.

Code: Select all

${IPT} -A INPUT -s 127.0.0.1 -j ACCEPT

You should inspect the incoming interface, not the address. All loopback traffic is on lo, regardless of the address used.

Code: Select all

${IPT} -A INPUT -s 192.168.0.0/24 -j ACCEPT

If you do not enable reverse path filtering, then you should use an interface qualifier here.

Code: Select all

${IPT} -A INPUT -p tcp -m multiport --dports 8022,8080,8443,64738 -j ACCEPT
${IPT} -A INPUT -p tcp -m multiport --dports 1069,3000,8888,8889,9418 -j ACCEPT

Why are these separate rules?

Code: Select all

${IPT} -A FORWARD -s 192.168.0.0/24 -j ACCEPT
${IPT} -A FORWARD -d 192.168.0.0/24 -j ACCEPT

Again, an interface qualifier may be needed.

[Akaihiryuu]

Ok testing done. With route-nopull and the iptables rules, inbound connections sort of work but they are extremely slow (as in, an HTTP test page I have takes minutes to load rather than seconds). This makes no sense at all. It should either work or not. To get everything to work properly, I still need to have it pull the routes and send all outbound traffic through the tunnel. This has to be a routing table problem but I don't know where to start. The only difference between the two states are in openvpn.conf, whether or not I'm using route-nopull. Everything else including iptables rules is identical. As far as the iptables save thing, I use iptables save for my permanent rules that don't involve the tunnel at all. I didn't really want to edit those, so I'm using a script to bring up alternate rules when I'm using the tunnel (I'm not expecting this situation to be permanent). The reason I'm setting policies back and deleting things is because the script can also completely flush out iptables, and I want to start from a known state. Also, my two input multiport rules are separate, because I have another configuration where the second set of ports is blocked off. The first set of ports I always want working, the second is more conditional and sometimes I turn them off. Thus I can comment out that rule, only leaving the first.

This is the routing table where everything works (but sends all outbound traffic through the tunnel which I do not want):

Code: Select all

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         178.170.139.193 128.0.0.0       UG    0      0        0 tun0
default         178.170.139.193 0.0.0.0         UG    0      0        0 tun0
default         192.168.101.1   0.0.0.0         UG    5      0        0 eth1
loopback        localhost       255.0.0.0       UG    0      0        0 lo
128.0.0.0       178.170.139.193 128.0.0.0       UG    0      0        0 tun0
178.170.139.2   192.168.101.1   255.255.255.255 UGH   0      0        0 eth1
178.170.139.192 0.0.0.0         255.255.255.192 U     0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
192.168.101.0   0.0.0.0         255.255.255.0   U     5      0        0 eth1

This is the routing table that has the weird semi-working condition:

Code: Select all

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.101.1   0.0.0.0         UG    5      0        0 eth1
loopback        localhost       255.0.0.0       UG    0      0        0 lo
178.170.139.192 0.0.0.0         255.255.255.192 U     0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
192.168.101.0   0.0.0.0         255.255.255.0   U     5      0        0 eth1

[chiefbag]

${NAT} -A POSTROUTING -p tcp -m multiport --sports 1069,3000,8888,8889,9418 -o tun0 -j MASQUERADE

Why are you MASQUERADEing based on sports?, that makes no sense.

[Akaihiryuu]

${NAT} -A POSTROUTING -p tcp -m multiport --sports 1069,3000,8888,8889,9418 -o tun0 -j MASQUERADE

Why are you MASQUERADEing based on sports?, that makes no sense.

Because I'm trying to use the tunnel for inbound connections. Thus I want anything connecting to those ports on my server to go back out via the tunnel. I don't want to use the tunnel for anything outbound (other than replies to connections on those ports obviously). I can't think of any way to match those other than sport, I obviously don't want to use dport because that would mess with my outbound traffic.

A summary of what I'm trying to do:

The internet I have is crappy and does not give me a public IP address. I'm using a VPN service to get a public IP address. I want to use that VPN for inbound services on those ports only. I want all of my regular outbound traffic to go out through my regular connection. The only thing I want going out through the tunnel are replies to inbound connections on those sports.

[chiefbag]

MASQURADE all traffic leaving tun0, it should solve a lot of your issues.

[Hu]

The machine-readable output produced by iptables-save can be saved anywhere. You can save alternate versions for VPN-up and VPN-down, then use iptables-restore to load an appropriate set. This would both let you start from a known state and avoid the race conditions I mentioned above.

[chiefbag]

Or at a minimum add the "-w" flag to your iptables command, still it won't guarantee the correct order.

Code: Select all

IPT="/sbin/iptables -w" 
NAT="/sbin/iptables -w -t nat" 

[chiefbag]

Or you could add your own lock mechanism per rule like the following or shove the rules into an array and loop through.

Code: Select all

${IPT} -A INPUT -p tcp -m multiport --dports 8022,8080,8443,64738 -j ACCEPT &
wait $!

[Hu]

That lock does not help with the race condition I meant. The lock prevents concurrent executions of iptables. My concern is that a packet which arrives while the script is executing will be processed with a partially built table that contains some rules from the script, but not all of them.

[chiefbag]

My concern is that a packet which arrives while the script is executing will be processed with a partially built table that contains some rules from the script, but not all of them.

Ya, I see what you are saying, using the iptables-restore command ensures all rules are loaded together on a per table basis, hence the COMMIT command if one looks at the output of iptables-save

[Akaihiryuu]

Nothing I can do in iptables is going to change which interface my traffic is going out on. Right now I have everything working as far as the VPN goes. But all traffic goes out of the VPN (which is what I don't want). This is the current routing table:

Code: Select all

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         jaina.multiplea 128.0.0.0       UG    0      0        0 tun0
default         jaina.multiplea 0.0.0.0         UG    0      0        0 tun0
default         192.168.101.1   0.0.0.0         UG    5      0        0 eth1
loopback        localhost       255.0.0.0       UG    0      0        0 lo
128.0.0.0       jaina.multiplea 128.0.0.0       UG    0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 br0
192.168.101.0   0.0.0.0         255.255.255.0   U     5      0        0 eth1
wordbox.doorope 192.168.101.1   255.255.255.255 UGH   0      0        0 eth1
paulthe.multipl 0.0.0.0         255.255.255.224 U     0      0        0 tun0

If I try to NAT stuff out through eth1 (my regular internet) with this routing table, the packets just never go anywhere. I've tested, and I cannot connect out at all. I HAVE to NAT out through tun0 with this routing table to get any sort of non-inbound connectivity. I don't know...I guess what I want to do may not be possible. I guess you just can't have two gateways.

[chiefbag]

Code: Select all

Kernel IP routing table 
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface 
0.0.0.0         jaina.multiplea 128.0.0.0       UG    0      0        0 tun0 
default         jaina.multiplea 0.0.0.0         UG    0      0        0 tun0

It looks like in your routing table all traffic will be routed out tun0 as it's the default gateway.

Why are you not using "route-nopull"?

[Akaihiryuu]

Code: Select all

Kernel IP routing table 
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface 
0.0.0.0         jaina.multiplea 128.0.0.0       UG    0      0        0 tun0 
default         jaina.multiplea 0.0.0.0         UG    0      0        0 tun0

It looks like in your routing table all traffic will be routed out tun0 as it's the default gateway.

Why are you not using "route-nopull"?

Because if I do that, then my internet connection works, but my inbound traffic via tun0 does not work.

[chiefbag]

You definitely can't leave your default route as tun0 so you either need to use the no-pull option ( I suggest to use this method ) or fix the routing table after to change back your default route.
You may need to manually configure a route for traffic to return through tun0 that originated there.

[Hu]

What did you observe that led you to believe you would not receive inbound traffic when your default route uses your ISP? The expected results in that case are that you can receive traffic over the VPN and that, if reverse path filtering is enabled, it will likely be dropped due to a bad reverse route. If reverse path filtering is disabled (not recommended), then it should respond over the main Internet connection, which won't produce a working virtual circuit, but will produce a response.

I expect that you need to route all responses to VPN-received traffic back over the VPN.

[Akaihiryuu]

You definitely can't leave your default route as tun0 so you either need to use the no-pull option ( I suggest to use this method ) or fix the routing table after to change back your default route.
You may need to manually configure a route for traffic to return through tun0 that originated there.

Yes, I the nopull option with a custom route for tun0 originated traffic would be best, but I have no clue how to do that.

[jamapii]

The problem is,

1. you want most outbound traffic not to go through VPN

2. but inbound connections and their reply traffic must go through VPN

1 is for performance, 2 is for necessity

You can't just apply rule 1 unconditionally, because anyone connecting to the public IP address expects replies to come from this public IP address. The server, that is reached via DNAT I guess, does not know this, and sends its replies directly, and they go their usual way bypassing VPN and DNAT.

There is no simple openvpn-only configuration that can achieve this.

I had this problem before, I hat cheap NAT routers with forced NAT inside the home network, but wanted a server to be reachable from outside the NAT router.

Server and VPN endpoint on the same box, the solution was (on the server box, vtun syntax):

Code: Select all

        ifconfig "%% 10.X.Y.2 pointopoint 10.X.Y.1 mtu 1450";   # necessary part of the vtun config
        ip "rule add priority 32700 from 10.X.Y.2 table 240";   
        ip "route add default via 10.X.Y.1 table 240";   
        ip "route flush cache";   

This takes advantage of server and vpn endpoint running on the same box. Whenever the server is contacted, its replies come from the tun interface with address 10.X.Y.2. This activates a special routing table, I chose 240, where everything gets routed through VPN, according to rule 2. above. Everything else uses the normal routing table, following rule 1. above.

Now if the server and the vpn are running on different boxes, this cannot be used. Still, a similar configuration is possible. The configuration must be on the VPN box. The "ip route"... lines can stay the same. (I have never tested or tried this)

The "ip rule" lines are different. They identify the traffic that is replies from the world-reachable servers and must therefore go through VPN.

If you don't mind everything from the server box to go through VPN, you can use "ip rule add ... from ...". This identifies traffic based on the source IP address (of the server).

To optimize more, you can use "ip rule add ... fwmark ..." and then add iptables rules to identify traffic based on source IP and source port (of the server). For a webserver, the source of its replies may be "-p tcp -s www --sport https". www must resolve to (or be replaced by) the webserver's IP address. Continue with e.g. "-j MARK --set-mark 123" on the iptables line.

Do not set the mark for the wrong ports or the wrong direction. Only the servers' reply packets must be marked. They go in the opposite direction of what is usually described in iptables filter rules.


Then, leave out redirect-private from openvpn config, and use the "ip rule" and "ip route" commands instead.

SNAT, as mentioned, might work too, but I avoided that when faced with the problem. I think in that case the traffic should SNAT or MASQUERADE, on the VPN endpoint, everything going _to_ the world-reachable servers. This rule should look something like "-p tcp -d www --dport https". Or just masquerade everything that is coming from tun (this may have undesirable side effects if the VPN has more purposes). However, I think this means the server never knows who is contacting it. Only the iptables system does.