Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
rsync tree verification: key update from closed networks
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
tazinblack
Veteran
Veteran


Joined: 23 Jan 2005
Posts: 1107
Location: Baden / Germany

PostPosted: Mon Jul 16, 2018 9:26 am    Post subject: rsync tree verification: key update from closed networks Reply with quote

Hello everyone,

I have a lot gentoo boxes running in a company network spread over the whole world. They have no direct connection to the internet and use the internal dns if any.
I update them via a proxy. This has worked fine for a lot of years.
But now I run into the problem that the portage key verification wants to update the gpg keys directly and therefor needs to resolve the ip of the keyserver.

Is there a designated way to do this using the proxy? I think I will not be the only one with this problem.
At the moment I use emerge-webrsync as workaround but this is very inefficient and blows up the bandwidth.
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes
Back to top
View user's profile Send private message
C5ace
Apprentice
Apprentice


Joined: 23 Dec 2013
Posts: 237
Location: Brisbane, Australia

PostPosted: Mon Jul 16, 2018 10:54 am    Post subject: Reply with quote

I look after 22 boxes in 1 local and 4 remote offices in Australia and one in Canary Islands.

Each office has a internet connection through a ADSL router and a always ON laptop running their local mail and DNS server in VirtualBox. The laptop's portage tree is updated daily by cron at around 5:00AM. The portage tree (/usr/portage) is accessed by the mix of local workstations through NFS when the workstations are powered up in the morning.

The actual update is done by ssh through the router -> laptop -> workstations by manually running a small script /opt/bin/upgrade and if necessary /opt/bin/cleanup.

/opt/bin/upgrade
Code:
#!/bin/bash

echo upgrade start

eselect news read
emerge --update --deep --with-bdeps=y --newuse --backtrack=300 --changed-deps=y --keep-going=y @world -va

echo upgrade end

/opt/bin/cleanup
Code:
#!/bin/sh

echo cleanup start

emerge --depclean
revdep-rebuild

echo cleanup end


I run the scripts every working day. First on my box and then on the remote boxes.

Change of use flags, kernel upgrades, etc. are done the traditional manual way through ssh.
Back to top
View user's profile Send private message
tazinblack
Veteran
Veteran


Joined: 23 Jan 2005
Posts: 1107
Location: Baden / Germany

PostPosted: Mon Jul 16, 2018 11:58 am    Post subject: Reply with quote

C5ace wrote:
I look after 22 boxes in 1 local and 4 remote offices in Australia and one in Canary Islands.

Each office has a internet connection through a ADSL router and a always ON laptop running their local mail and DNS server in VirtualBox. The laptop's portage tree is updated daily by cron at around 5:00AM. The portage tree (/usr/portage) is accessed by the mix of local workstations through NFS when the workstations are powered up in the morning.

The actual update is done by ssh through the router -> laptop -> workstations by manually running a small script /opt/bin/upgrade and if necessary /opt/bin/cleanup.

/opt/bin/upgrade
Code:
#!/bin/bash

echo upgrade start

eselect news read
emerge --update --deep --with-bdeps=y --newuse --backtrack=300 --changed-deps=y --keep-going=y @world -va

echo upgrade end

/opt/bin/cleanup
Code:
#!/bin/sh

echo cleanup start

emerge --depclean
revdep-rebuild

echo cleanup end


I run the scripts every working day. First on my box and then on the remote boxes.

Change of use flags, kernel upgrades, etc. are done the traditional manual way through ssh.


I think you do not understand my problem. The only possible way to reach the internet from inside is to use the proxy and there is no internal name resolution of servernames of the 'outside world'.
And by now the proxy is not able to handle hkps. It only supports http, https and rsync. And even if the proxy would do that, how would I have to tell emerge or eix to use the proxy for hkps?

So, in other words, the thing I need is a way to use the proxy for updating the gpg keys.
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes
Back to top
View user's profile Send private message
C5ace
Apprentice
Apprentice


Joined: 23 Dec 2013
Posts: 237
Location: Brisbane, Australia

PostPosted: Mon Jul 16, 2018 1:23 pm    Post subject: Reply with quote

Quote:

I think you do not understand my problem. The only possible way to reach the internet from inside is to use the proxy and there is no internal name resolution of servernames of the 'outside world'.
And by now the proxy is not able to handle hkps. It only supports http, https and rsync. And even if the proxy would do that, how would I have to tell emerge or eix to use the proxy for hkps?

So, in other words, the thing I need is a way to use the proxy for updating the gpg keys.


Consider my laptop running Gentoo to function as your proxy server and at the same time the local (via NFS) /usr/portage/* directory of the LAN workstations. The gpgkeys are updated when 'emerge --sync' is run on my laptop (your proxy server) and is automatically available on all workstations.

ADSL here is slow, between 500Kbit/s and 10Mbit/s down speed. I use this network layout:
WAN -> ADSL modem/NAT/Firewall -> LAN
-> laptop with NFS shared /usr/portage/*
-> Workstations
-> local DNS and web server

The ssh connection is trough port forwarding through the ADSL modem/NAT/Firewall and the laptop firewall. Then from the laptop by ssh to each workstation.

Provide details of your the network layout.

Guss von Brisbane
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6220

PostPosted: Mon Jul 16, 2018 8:05 pm    Post subject: Reply with quote

You could patch portage to remove the code which fetches the key.
You could also write an enhancement request on bugzilla to introduce an option to skip this step.
Of course, in both cases you should be aware about the reason why portage does this now by default: In case of an emergency, gentoo infra can revoke the key; if you do not verify, you take away that possibly last chance in case of a major hack. On the other hand, IMHO your reasons are legitimate. I could imagine that Zac might agree to introduce such an option if the recommendation of using this option can be communicated clear enough (e.g. skip-key-revoke-check-I-understand-that-this-is-risky=yes)
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 6841

PostPosted: Mon Jul 16, 2018 8:59 pm    Post subject: Reply with quote

i don't think user wish skip the key check, he just want the key check to works using http, https or rsync, which would allow his proxy to handle it.
that's something still for bugs.gentoo.org
Back to top
View user's profile Send private message
Aiken
Apprentice
Apprentice


Joined: 22 Jan 2003
Posts: 209
Location: Toowoomba/Australia

PostPosted: Mon Jul 16, 2018 10:04 pm    Post subject: Re: rsync tree verification: key update from closed networks Reply with quote

tazinblack wrote:

But now I run into the problem that the portage key verification wants to update the gpg keys directly and therefor needs to resolve the ip of the keyserver.


There is an option for turning off that check. Are you in a position where you could update the tree on one machine and have the other machines update their tree from it? What I have is more a soho setup and everything is in the same building. I have the master copy updated from a cron job which has the verification turned on and is running rsynd. All the other machines have verification turned off and sync from my local master copy.
_________________
Beware the grue.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6220

PostPosted: Tue Jul 17, 2018 6:18 am    Post subject: Re: rsync tree verification: key update from closed networks Reply with quote

Aiken wrote:
There is an option for turning off that check.

Currently not. You can only turn on/off signature checking as a whole, but if you want signature checking the update of the key is mandatory.
Back to top
View user's profile Send private message
tazinblack
Veteran
Veteran


Joined: 23 Jan 2005
Posts: 1107
Location: Baden / Germany

PostPosted: Tue Jul 17, 2018 8:29 am    Post subject: Reply with quote

krinn wrote:
i don't think user wish skip the key check, he just want the key check to works using http, https or rsync, which would allow his proxy to handle it.
that's something still for bugs.gentoo.org


That's correct. I don't want to skip the check. I just need it to work via proxy.
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes
Back to top
View user's profile Send private message
tazinblack
Veteran
Veteran


Joined: 23 Jan 2005
Posts: 1107
Location: Baden / Germany

PostPosted: Tue Jul 17, 2018 8:42 am    Post subject: Reply with quote

[quote="C5ace"]
Quote:
...
The ssh connection is trough port forwarding through the ADSL modem/NAT/Firewall and the laptop firewall. Then from the laptop by ssh to each workstation.
...


Our security guys would bite my head off, if I would try to do thinks like that.

Sorry but we are very restrictive with security. That's also a reason why I want/need the tree verification.
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes
Back to top
View user's profile Send private message
tazinblack
Veteran
Veteran


Joined: 23 Jan 2005
Posts: 1107
Location: Baden / Germany

PostPosted: Tue Jul 17, 2018 8:59 am    Post subject: Reply with quote

https://bugs.gentoo.org/show_bug.cgi?id=661376
_________________
Gruß / Regards
tazinblack
_______________________________________________________
what's the point in being grown up if you can't be childish sometimes
Back to top
View user's profile Send private message
Aiken
Apprentice
Apprentice


Joined: 22 Jan 2003
Posts: 209
Location: Toowoomba/Australia

PostPosted: Tue Jul 17, 2018 9:01 am    Post subject: Re: rsync tree verification: key update from closed networks Reply with quote

mv wrote:
Aiken wrote:
There is an option for turning off that check.

Currently not.


The relevant news items lists "sync-rsync-verify-metamanifest = no" as the option to turn it off. With that set no waiting for a refresh of the keys and no verification.

Hence what I have set up with one machine doing the sync from the internet and doing the refresh and verify with the other 13 having that option set in /etc/portage/repos.conf/gentoo.conf so they miss out on the slow refresh and at times even slower verification when they sync from the first machine.
_________________
Beware the grue.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6220

PostPosted: Tue Jul 17, 2018 9:32 am    Post subject: Re: rsync tree verification: key update from closed networks Reply with quote

Aiken wrote:
With that set no waiting for a refresh of the keys and no verification.

I had git verification in mind. But from your description, it appears that for rsync the situation is similar: I emphasized the relevant part as I understand it now: Not only the keys fail to be verified but also the signature (of the manifest). (For git, I verified it in the portage source code; for rsync a quick glance at the source was not enough to understand exactly).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum