View previous topic :: View next topic |
Author |
Message |
Wizumwalt Guru
Joined: 20 Aug 2006 Posts: 547
|
Posted: Tue May 18, 2021 10:15 pm Post subject: Allow pass-through of git from guest via host? |
|
|
I'm trying to add the needed rules to allow a guest machine (using QEMU) to access git repos via a host machine. I've tried the following to no avail.
The error I currently get on a virtual guest machine is:
Code: | $ git pull
ssh: connect to host gitlab.com port 22: Network is unreachable
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
|
Of course if I turn the firewall off on the host machine, the above error disappears and I can access my git repos. My goal is to leave my firewall up. I have the following rules for git and dns.
Code: | $ sudo iptables-save -c
# Generated by iptables-save v1.8.7 on Tue May 18 17:08:57 2021
*nat
:PREROUTING ACCEPT [484:31281]
:INPUT ACCEPT [4:224]
:OUTPUT ACCEPT [6245:393232]
:POSTROUTING ACCEPT [1923:115380]
:DOCKER - [0:0]
[10:733] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
[0:0] -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
[0:0] -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
[2653067:176713609] -A POSTROUTING -o br0 -j MASQUERADE
[0:0] -A DOCKER -i docker0 -j RETURN
COMMIT
# Completed on Tue May 18 17:08:57 2021
# Generated by iptables-save v1.8.7 on Tue May 18 17:08:57 2021
*mangle
:PREROUTING ACCEPT [693156427:577045487522]
:INPUT ACCEPT [673998326:557852638205]
:FORWARD ACCEPT [17644491:19019848649]
:OUTPUT ACCEPT [411232509:43705530305]
:POSTROUTING ACCEPT [428659269:62690232552]
COMMIT
# Completed on Tue May 18 17:08:57 2021
# Generated by iptables-save v1.8.7 on Tue May 18 17:08:57 2021
*filter
:INPUT DROP [0:0]
:FORWARD DROP [10:600]
:OUTPUT ACCEPT [1133:121152]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
[1075575:703529026] -A INPUT -i lo -j ACCEPT
[1623:246726] -A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m state --state NEW -j DROP
[579377134:469553270886] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
[20:1184] -A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT
[70:4730] -A INPUT -p udp -m udp --dport 53 -j ACCEPT
[0:0] -A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
[0:0] -A INPUT -p tcp -m tcp --dport 443 -m state --state NEW -j ACCEPT
[0:0] -A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
[562668:30070661] -A INPUT -j DROP
[504:32756] -A FORWARD -j DOCKER-USER
[504:32756] -A FORWARD -j DOCKER-ISOLATION-STAGE-1
[0:0] -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -o docker0 -j DOCKER
[0:0] -A FORWARD -i docker0 ! -o docker0 -j ACCEPT
[0:0] -A FORWARD -i docker0 -o docker0 -j ACCEPT
[12:672] -A FORWARD -p udp -m udp --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
[24:1872] -A FORWARD -p udp -m udp --sport 53 -m state --state ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -p tcp -m tcp --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
[0:0] -A FORWARD -p tcp -m tcp --sport 53 -m state --state ESTABLISHED -j ACCEPT
[1075575:703529026] -A OUTPUT -o lo -j ACCEPT
[330767:27732292] -A OUTPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
[0:0] -A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
[504:32756] -A DOCKER-ISOLATION-STAGE-1 -j RETURN
[0:0] -A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
[0:0] -A DOCKER-ISOLATION-STAGE-2 -j RETURN
[16997514:18464885695] -A DOCKER-USER -j RETURN
COMMIT
# Completed on Tue May 18 17:08:57 2021
|
And here are my interfaces ...
Code: |
$ ifconfig
br0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST> mtu 1500
inet 192.168.10.10 netmask 255.255.255.0 broadcast 192.168.10.255
inet6 fd9d:93f2:ab6a:0:6c8d:7aff:fa32:b487 prefixlen 64 scopeid 0x0<global>
inet6 fe80::6c8d:7aff:fe32:b487 prefixlen 64 scopeid 0x20<link>
ether 6e:8d:7a:32:b4:87 txqueuelen 1000 (Ethernet)
RX packets 179036 bytes 132971649 (126.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 169986 bytes 17037083 (16.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:61:c8:e3:9f txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::96de:80ff:feb4:7bb6 prefixlen 64 scopeid 0x20<link>
ether 94:de:80:b4:7a:a6 txqueuelen 1000 (Ethernet)
RX packets 180539 bytes 136260398 (129.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 170057 bytes 17763388 (16.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 20 memory 0xf3100000-f3120000
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 3484 bytes 183703 (179.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3484 bytes 183703 (179.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
tap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::6d8d:7aaf:fe32:b487 prefixlen 64 scopeid 0x20<link>
ether 6e:8d:7a:32:a4:27 txqueuelen 1000 (Ethernet)
RX packets 296 bytes 43010 (42.0 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 387 bytes 39088 (38.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
tap1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::c035:e1ff:fe9c:4f17 prefixlen 64 scopeid 0x20<link>
ether c2:35:e1:9c:af:29 txqueuelen 1000 (Ethernet)
RX packets 314 bytes 50810 (49.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 369 bytes 37400 (36.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
|
Any help appreciated.
Last edited by Wizumwalt on Sun May 23, 2021 11:17 pm; edited 1 time in total |
|
Back to top |
|
|
Ralphred Guru
Joined: 31 Dec 2013 Posts: 495
|
Posted: Tue May 18, 2021 11:16 pm Post subject: |
|
|
Do you need only this container to access the outside world, or all of them?
If it's only "this one" we need the ip from inside the container so we can set a "-s"
You only need to use dns and git from inside the container?
No outside to inside traffic other than that which initiated from inside?
Do you want to use the firewall outside of controlling this(ese) container(s)?
I'm just going to dump this here for reference a bit later...And finally, can you post the output of the following please. Code: | iptables --line-numbers -n -v -L
iptables --line-numbers -n -v -L -t nat | It'll be easier to work with than docker sending packets all over the place to empty chains. |
|
Back to top |
|
|
Wizumwalt Guru
Joined: 20 Aug 2006 Posts: 547
|
Posted: Tue May 18, 2021 11:34 pm Post subject: |
|
|
Quote: | Do you need only this container to access the outside world, or all of them? |
I need both (all) of them to access git repos.
Quote: | You only need to use dns and git from inside the container? |
Yes, correct, just these two services and portage. However, the latter already is able to access its repos for doing updates.
Quote: | No outside to inside traffic other than that which initiated from inside? |
Correct.
Quote: | Do you want to use the firewall outside of controlling this(ese) container(s)? |
Currently I have the guest machine firewalls turned off. I'm ok with this for simplicity as long as I have the host firewall running.
And here's "$ sudo iptables --line-numbers -n -v -L > /tmp/fw"
Code: | Chain INPUT (policy DROP 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 1076K 704M ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
2 1623 247K DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp flags:!0x17/0x02 state NEW
3 579M 470G ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
4 20 1184 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 state NEW
5 70 4730 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:53
6 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 state NEW
7 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 state NEW
8 0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 icmptype 0
9 563K 30M DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy DROP 10 packets, 600 bytes)
num pkts bytes target prot opt in out source destination
1 504 32756 DOCKER-USER all -- * * 0.0.0.0/0 0.0.0.0/0
2 504 32756 DOCKER-ISOLATION-STAGE-1 all -- * * 0.0.0.0/0 0.0.0.0/0
3 0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
4 0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
7 12 672 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 state NEW,ESTABLISHED
8 24 1872 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp spt:53 state ESTABLISHED
9 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 state NEW,ESTABLISHED
10 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp spt:53 state ESTABLISHED
Chain OUTPUT (policy ACCEPT 24566 packets, 2317K bytes)
num pkts bytes target prot opt in out source destination
1 1076K 704M ACCEPT all -- * lo 0.0.0.0/0 0.0.0.0/0
2 331K 28M ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 icmptype 8
Chain DOCKER (1 references)
num pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
2 504 32756 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DROP all -- * docker0 0.0.0.0/0 0.0.0.0/0
2 0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
num pkts bytes target prot opt in out source destination
1 17M 18G RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
|
And ... "iptables --line-numbers -n -v -L -t nat"
Code: |
Chain PREROUTING (policy ACCEPT 484 packets, 31281 bytes)
num pkts bytes target prot opt in out source destination
1 10 733 DOCKER all -- * * 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 4 packets, 224 bytes)
num pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 7261 packets, 456K bytes)
num pkts bytes target prot opt in out source destination
1 0 0 DOCKER all -- * * 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 2163 packets, 130K bytes)
num pkts bytes target prot opt in out source destination
1 0 0 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0
2 2654K 177M MASQUERADE all -- * br0 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (2 references)
num pkts bytes target prot opt in out source destination
1 0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0 |
|
|
Back to top |
|
|
Ralphred Guru
Joined: 31 Dec 2013 Posts: 495
|
Posted: Wed May 19, 2021 12:10 am Post subject: |
|
|
Code: | iptables -N dockerwall
iptables -A dockerwall -p tcp -m tcp --dport 22 -m state --state NEW -m comment --comment "From container to ssh" -j ACCEPT
iptables -A dockerwall -p tcp -m tcp --dport 53 -m state --state NEW -m comment --comment "From container to dns" -j ACCEPT
#edit
iptables -A dockerwall -p udp -m udp --dport 53 -m state --state NEW -m comment --comment "From container to dns" -j ACCEPT
#endedit
iptables -A dockerwall -p tcp -m tcp --dport 80 -m state --state NEW -m comment --comment "From container to http" -j ACCEPT
iptables -A dockerwall -p tcp -m tcp --dport 443 -m state --state NEW -m comment --comment "From container to https" -j ACCEPT
iptables -A dockerwall -p tcp -m tcp --dport 9418 -m state --state NEW -m comment --comment "From container to git" -j ACCEPT
iptables -A dockerwall -m comment --comment "Log spurious packets" -j LOG --log-prefix "NaughtyDocker: " --log-level 6
iptables -A dockerwall -m comment --comment "Drop after logging" -j DROP
iptables -I INPUT 4 -i docker0 -m comment --comment "Anything from containers gets sent to dockerwall" -j dockerwall
iptables -I FORWARD 1 -i docker0 -m comment --comment "Anything from containers gets sent to dockerwall" -j dockerwall
iptables -I FORWARD 2 -m state --state RELATED,ESTABLISHED -m comment --comment "Let stuff we asked for back in" -j ACCEPT |
This is the "firewall" part done, the NAT stuff "looks OK", you should only need the MASQUERADE rule...
Last edited by Ralphred on Wed May 19, 2021 12:54 am; edited 1 time in total |
|
Back to top |
|
|
Ralphred Guru
Joined: 31 Dec 2013 Posts: 495
|
Posted: Wed May 19, 2021 12:49 am Post subject: |
|
|
If something doesn't work you should be able to grep for "NaughtyDocker" in /var/log/messages and see where it went wrong.
If you want to drop the log rule when it's all safe and sound, use the Code: | iptables --line-numbers -n -v -L | to get the rule number (first field) then Code: | iptables -D dockerwall [rule number] | to delete it.
EDIT: Forgot DNS needs UDP too... will edit above... |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21607
|
Posted: Wed May 19, 2021 1:58 am Post subject: |
|
|
Ralphred wrote: | And finally, can you post the output of the following please. Code: | iptables --line-numbers -n -v -L
iptables --line-numbers -n -v -L -t nat | It'll be easier to work with than docker sending packets all over the place to empty chains. | The original output, from iptables-save, is better. It's more verbose, but it shows exactly what is loaded. For example, OP might have had rules in the mangle table that were relevant, and your commands would have hidden those completely.
OP talks about using virtual machines, and I see tap interfaces in the ifconfig output. Therefore, I think Docker may be completely irrelevant here. OP: to confirm, what exactly do you run to "turn the firewall off on the host machine"? /etc/init.d/iptables stop?
Network unreachable is a bit strange to be triggered by a firewall, particularly this firewall. I have seen a firewall with the right options trigger that, I think, but the shown rules do not have the directives I would expect to trigger that error. I see that gitlab.com has a AAAA record. Are you sure the guest is using IPv4 for the failed connection? What is the output of ip6tables-save -c? |
|
Back to top |
|
|
Ralphred Guru
Joined: 31 Dec 2013 Posts: 495
|
Posted: Wed May 19, 2021 3:17 am Post subject: |
|
|
Hu wrote: | The original output, from iptables-save, is better. |
It really isn't, especially when you are trying to follow chains. There are no -t mangle rules defined, hence its missing from the request.
What we are interested in in getting in front of the over complicated mess left by docker, which involves seeing rule numbers.
If there is some "docker-firewall [command]" wrapper then I can understand having set chains to follow, but peoples (even devs) proclivity to ignore that --comment exists irks me, especially when my iptables-save output is ~750 lines long.
Hu wrote: | Network unreachable is a bit strange to be triggered by a firewall | Again no, without the overload rule on the "output to LAN" interface there is no SNAT, all the "replied to requests" by outside severs hit empty/closed ports on VM host. If you "/etc/init.d/iptables stop" the whole thing gets flushed, along with the SNAT. |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21607
|
Posted: Wed May 19, 2021 3:27 am Post subject: |
|
|
Ralphred wrote: | Hu wrote: | The original output, from iptables-save, is better. | It really isn't, especially when you are trying to follow chains. There are no -t mangle rules defined, hence its missing from the request. | If you mean missing from what you asked OP to repeat, then yes. But if OP had started with just iptables --list, how would you have known that his mangle table was empty? I always encourage requesters to provide their full iptables data, and iptables-save -c is easier to type, and easier for them to remember, than iptables -Lnvx --line-numbers -t table for every table supported in their kernel.
Following chains is just as easy in one as the other. All rules for a given chain are grouped together in the iptables-save output, just as they are in the iptables --list output. Ralphred wrote: | What we are interested in in getting in front of the over complicated mess left by docker, which involves seeing rule numbers. | How do rule numbers help us here? Those are a minor convenience once we have identified a rule that we want to tell the OP to change, but I don't see how they help us diagnose the problem. Ralphred wrote: | Hu wrote: | Network unreachable is a bit strange to be triggered by a firewall | Again no, without the overload rule on the "output to LAN" interface there is no SNAT, all the "replied to requests" by outside severs hit empty/closed ports on VM host. | Without source address rewriting, the responses from outside will go to the private address the VM claimed to have, which will almost certainly not route back to the VM. That won't cause the VM to report "Network is unreachable" though. It just means connectivity doesn't work. Ralphred wrote: | If you "/etc/init.d/iptables stop" the whole thing gets flushed, along with the SNAT. | OP reports that disabling the firewall makes this work. You say that disabling the firewall will remove SNAT, at which point the outside responses will hit "empty/closed ports". Wouldn't that then be the failure mode, not the success mode? How does removing SNAT make things better? |
|
Back to top |
|
|
Ralphred Guru
Joined: 31 Dec 2013 Posts: 495
|
Posted: Wed May 19, 2021 4:32 am Post subject: |
|
|
Hu wrote: | But if OP had started with just iptables --list, how would you have known that his mangle table was empty? |
OP didn't start with iptables --list he started with iptables-save -c.In this case having the info repeated in the way requested made it simple to fix the firewall to fit OP's use case. "But what if" rhetorical situations that do not exist here are pointless exercises.
Hu wrote: | Without source address rewriting, the responses from outside will go to the private address the VM claimed to have, which will almost certainly not route back to the VM. That won't cause the VM to report "Network is unreachable" though. OP reports that disabling the firewall makes this work. |
You're making the assumption docker doesn't mess with the firewall after is stopped, the fact the container does work means something is messing with it. But this is again entirely academic because OP's use case was to make the container work with the firewall "on". With the assumption that docker will continue mess with things, a custom table with only 3 insert commands ahead of docker in the overall chain is the first step to circumventing dockers efforts, stepping past the FORWARD tables DROP policy and providing PoC for a sustainable solution. If/when it doesn't work, then looking at the routing with the firewall off and checking /proc/net/nf_conntrack would be the next step. |
|
Back to top |
|
|
Wizumwalt Guru
Joined: 20 Aug 2006 Posts: 547
|
Posted: Sun May 23, 2021 11:09 pm Post subject: |
|
|
Hu wrote: | OP talks about using virtual machines, and I see tap interfaces in the ifconfig output. Therefore, I think Docker may be completely irrelevant here. OP: to confirm, what exactly do you run to "turn the firewall off on the host machine"? /etc/init.d/iptables stop
|
Sorry for the late reply. I was kind of thinking the same thing. The docker stuff confused me as I use virtual machines (qemu). To turn the firewall off, I use /etc/init.d/iptables stop
Hu wrote: | What is the output of ip6tables-save -c? |
Code: |
$ sudo ip6tables-save -c
# Generated by ip6tables-save v1.8.7 on Sun May 23 18:05:55 2021
*mangle
:PREROUTING ACCEPT [1684:231888]
:INPUT ACCEPT [1131:180416]
:FORWARD ACCEPT [2254:356456]
:OUTPUT ACCEPT [418:24048]
:POSTROUTING ACCEPT [2672:380504]
COMMIT
# Completed on Sun May 23 18:05:55 2021
# Generated by ip6tables-save v1.8.7 on Sun May 23 18:05:55 2021
*filter
:INPUT ACCEPT [1131:180416]
:FORWARD ACCEPT [2254:356456]
:OUTPUT ACCEPT [418:24048]
COMMIT
# Completed on Sun May 23 18:05:55 2021
|
|
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21607
|
Posted: Mon May 24, 2021 12:18 am Post subject: |
|
|
IPv6 looks fine. I don't see anything obviously wrong with the IPv4 setup, but mixing bridging and masquerading is a bit strange. What interfaces are part of the bridge? Why are you masquerading traffic that goes out of the bridge? Have you tried selectively removing firewall rules to find which rule(s) result in the failure? |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|