Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Portage rsync tree verification
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
uberDoward
n00b
n00b


Joined: 09 Jun 2011
Posts: 46

PostPosted: Fri Jun 08, 2018 8:24 pm    Post subject: Reply with quote

Is this related? Suddenly I can't sync:

Code:
>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys from keyserver ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: Server indicated a failure


[Moderator edit: added [code] tags to preserve output layout. -Hu]
Back to top
View user's profile Send private message
uberDoward
n00b
n00b


Joined: 09 Jun 2011
Posts: 46

PostPosted: Mon Jun 11, 2018 9:54 pm    Post subject: Reply with quote

Here's what I'm seeing as of today:

Code:
>>> Syncing repository 'gentoo' into '/usr/portage'...
 * Using keys from /usr/share/openpgp-keys/gentoo-release.asc
 * Refreshing keys from keyserver ...OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: Server indicated a failure

OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: Server indicated a failure
Back to top
View user's profile Send private message
MTZ
Tux's lil' helper
Tux's lil' helper


Joined: 13 Jul 2002
Posts: 90
Location: Germany - near FFM

PostPosted: Wed Jul 04, 2018 12:14 pm    Post subject: Reply with quote

I had the same issue but after updating

Code:
 app-crypt/openpgp-keys-gentoo-release-20180703::gentoo


the error was gone.
Back to top
View user's profile Send private message
ayeyes
Tux's lil' helper
Tux's lil' helper


Joined: 03 Dec 2017
Posts: 104

PostPosted: Wed Jul 04, 2018 3:35 pm    Post subject: Reply with quote

Yesterday and today:

Code:
sent 61.57K bytes  received 11.31M bytes  429.13K bytes/sec
total size is 218.80M  speedup is 19.24
!!! Manifest verification failed:
OpenPGP verification failed:
gpg: Signature made Wed 04 Jul 2018 14:38:30 UTC
gpg:                using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
gpg: Can't check signature: No public key

q: Updating ebuild cache for /usr/portage ...
q: Finished 35635 entries in 0.152047 seconds

Action: sync for repo: gentoo, returned code = 1


Portage 2.3.41 (python 3.5.5-final-0, default/linux/amd64/17.0/desktop, gcc-7.3.0, glibc-2.27-r5, 4.17.3-gentoo x86_64)
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA

PostPosted: Wed Jul 04, 2018 3:58 pm    Post subject: Reply with quote

ayeyes wrote:
Yesterday and today:

Code:
sent 61.57K bytes  received 11.31M bytes  429.13K bytes/sec
total size is 218.80M  speedup is 19.24
!!! Manifest verification failed:
OpenPGP verification failed:
gpg: Signature made Wed 04 Jul 2018 14:38:30 UTC
gpg:                using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
gpg: Can't check signature: No public key

q: Updating ebuild cache for /usr/portage ...
q: Finished 35635 entries in 0.152047 seconds

Action: sync for repo: gentoo, returned code = 1


Portage 2.3.41 (python 3.5.5-final-0, default/linux/amd64/17.0/desktop, gcc-7.3.0, glibc-2.27-r5, 4.17.3-gentoo x86_64)
I am seeing this too. Is this related the github hack (as in keys have been revoked)?

Is the tree in suspect state at this time?
Back to top
View user's profile Send private message
ayeyes
Tux's lil' helper
Tux's lil' helper


Joined: 03 Dec 2017
Posts: 104

PostPosted: Wed Jul 04, 2018 4:23 pm    Post subject: Reply with quote

MTZ wrote:
I had the same issue but after updating

Code:
 app-crypt/openpgp-keys-gentoo-release-20180703::gentoo


the error was gone.


Thank you! That fixed my problem too. I downloaded the latest snapshot from a mirror. Verified it with gpg. Extracted it. Deleted the files in '/usr/portage/app-crypt/openpgp-keys-gentoo-release/', and copied the files from the snapshot I extracted in /Downloads/. Then updated to 'openpgp-keys-gentoo-release-20180703'. and the --sync verification worked.
Back to top
View user's profile Send private message
desultory
Bodhisattva
Bodhisattva


Joined: 04 Nov 2005
Posts: 9410

PostPosted: Thu Jul 05, 2018 2:50 am    Post subject: Reply with quote

devsk wrote:
ayeyes wrote:
Yesterday and today:

Code:
sent 61.57K bytes  received 11.31M bytes  429.13K bytes/sec
total size is 218.80M  speedup is 19.24
!!! Manifest verification failed:
OpenPGP verification failed:
gpg: Signature made Wed 04 Jul 2018 14:38:30 UTC
gpg:                using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
gpg: Can't check signature: No public key

q: Updating ebuild cache for /usr/portage ...
q: Finished 35635 entries in 0.152047 seconds

Action: sync for repo: gentoo, returned code = 1


Portage 2.3.41 (python 3.5.5-final-0, default/linux/amd64/17.0/desktop, gcc-7.3.0, glibc-2.27-r5, 4.17.3-gentoo x86_64)
I am seeing this too. Is this related the github hack (as in keys have been revoked)?
So far as I know, no relevant keys have been revoked.
devsk wrote:
Is the tree in suspect state at this time?
No.

As noted by MTZ, just update to the latest app-crypt/openpgp-keys-gentoo-release and it should finish successfully.
Back to top
View user's profile Send private message
fb
l33t
l33t


Joined: 08 Dec 2003
Posts: 636
Location: New Zealand

PostPosted: Thu Jul 05, 2018 9:50 pm    Post subject: Reply with quote

desultory wrote:
devsk wrote:
ayeyes wrote:
Yesterday and today:

Code:
sent 61.57K bytes  received 11.31M bytes  429.13K bytes/sec
total size is 218.80M  speedup is 19.24
!!! Manifest verification failed:
OpenPGP verification failed:
gpg: Signature made Wed 04 Jul 2018 14:38:30 UTC
gpg:                using RSA key E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
gpg: Can't check signature: No public key

q: Updating ebuild cache for /usr/portage ...
q: Finished 35635 entries in 0.152047 seconds

Action: sync for repo: gentoo, returned code = 1


Portage 2.3.41 (python 3.5.5-final-0, default/linux/amd64/17.0/desktop, gcc-7.3.0, glibc-2.27-r5, 4.17.3-gentoo x86_64)
I am seeing this too. Is this related the github hack (as in keys have been revoked)?
So far as I know, no relevant keys have been revoked.
devsk wrote:
Is the tree in suspect state at this time?
No.

As noted by MTZ, just update to the latest app-crypt/openpgp-keys-gentoo-release and it should finish successfully.


OK I have seen that exact message since yesterday but I am already up to date with app-crypt/openpgp-keys-gentoo-release (20180702). Should I re-install?
Back to top
View user's profile Send private message
fb
l33t
l33t


Joined: 08 Dec 2003
Posts: 636
Location: New Zealand

PostPosted: Thu Jul 05, 2018 11:40 pm    Post subject: Reply with quote

I just discovered that I wasn't up to date. The eix database was just not updated because it is done after validation of the tree. This is a bit of a chicken and egg problem if you need to have a certain version of the key before being able to validate.
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1179

PostPosted: Sun Jul 08, 2018 10:31 am    Post subject: Reply with quote

I too have had very spotty luck with the basic use of either -sync in emerge which I don't understand what could be causing my specific error which is slightly different from others in that it usually kicks out something like "Connection impossible" or something to that effect.

If I try 4 or 5 times in a row then it works. I am super frustrated by this, and there appears to be nothing that I can upgrade. I am thinking maybe because I moved my OS to another disk (virtual) that it could be related to that, but I am really not sure. It seems like it could be tied to almost anything, and right now it is basically stuck on the Verifying /usr/portage step.

Do you think I should reinstall portage, or maybe change some settings somewhere in /etc perhaps?
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Sun Jul 08, 2018 10:38 am    Post subject: Reply with quote

you could disable it, and wait for it to mature enough.
USE="-rsync-verify" emerge -1 portage
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Sun Jul 08, 2018 1:33 pm    Post subject: Reply with quote

krinn wrote:
you could disable it, and wait for it to mature enough.

krinn ... in my case that may be forever ... as the dependency on app-crypt/gnupg-2.x would require I abandon (the soon to be removed, though supported upstream) =app-crypt/gnupg-1.4.21. In short, I either accept all the inconviences of gnupg-2 (such as pinentry breaking my use case) or stay with USE="-rsync-verify" indefinitely.

best ... khay
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Sun Jul 08, 2018 3:18 pm    Post subject: Reply with quote

khayyam wrote:
krinn ... in my case that may be forever ...

actually for all my hosts, it will, it's slower and not need, as i use only one host to sync and use this one to let other sync to it.
so only that one will have the feature, which is for now disable, because the feature looks too buggy for now.
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1179

PostPosted: Mon Jul 09, 2018 8:57 pm    Post subject: Reply with quote

So khayyam does this mean adding the flag to global use variable in make.conf, and can that be done? Or else should I enter it in package.use/ folder
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Mon Jul 09, 2018 10:11 pm    Post subject: Reply with quote

LIsLinuxIsSogood wrote:
So khayyam does this mean adding the flag to global use variable in make.conf, and can that be done? Or else should I enter it in package.use/ folder

LIsLinuxIsSogood ... it's best placed in package.use.

best ... khay
Back to top
View user's profile Send private message
jackwolf
n00b
n00b


Joined: 08 Jul 2018
Posts: 3

PostPosted: Wed Jul 11, 2018 11:36 am    Post subject: Reply with quote

I activate the "rsync-verify" for portage-2.3.42 and reemerge.

I have the /etc/portage/repos.conf/gentoo.conf

Code:
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /usr/portage
sync-type = rsync
sync-uri = rsync://rsync.gentoo.org/gentoo-portage
auto-sync = yes
sync-rsync-verify-jobs = 1
sync-rsync-verify-metamanifest = no
sync-rsync-verify-max-age = 24
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
sync-openpgp-key-refresh-retry-count = 40
sync-openpgp-key-refresh-retry-overall-timeout = 1200
sync-openpgp-key-refresh-retry-delay-exp-base = 2
sync-openpgp-key-refresh-retry-delay-max = 60
sync-openpgp-key-refresh-retry-delay-mult = 4


No other mirrors in make.conf.

But if i emerge --sync i don't get any message about pgp verify.
If i do "emato verify -K /usr/share/openpgp-keys/gentoo-release.asc /usr/portage"

Code:

INFO:root:Refreshing keys from keyserver...
INFO:root:Keys refreshed.
INFO:root:Manifest timestamp: 2018-07-10 12:38:39 UTC
INFO:root:Valid OpenPGP signature found:
INFO:root:- primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D
INFO:root:- subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
INFO:root:- timestamp: 2018-07-10 12:38:39 UTC
INFO:root:Verifying /usr/portage...
ERROR:root:Manifest mismatch for metadata/news/Manifest
  BLAKE2B: expected: cd4c58254f8d8d80b151cb3530d4e5407db38c02fec04e83fc5890d0ff594427e40571e1c9c0ddcdc71543bb65c7e4033f1a002659bf9ad14c406ca5c4f0ad96, have
d2182f89697402f47205cdec571634f5f65f5366d9f87b6350a50ea2a9313d0c7a82af1a1edf2ec19e255926b06ee2e
  SHA512: expected: 1e4a07162046e8d3be561fc036ef7a16466dd9ff3bfeab934af536eff2588b8021c01b47599bcc3a7575f93701e7cfbfb6a620ec3143495599d13c4ce0506bb7, have:
2eb4ab220780ca93d909514fbfcdf03aa85394349192617f8f36f5236f7d6c9345c72be789419a7ff2a5800aed5108



What is wrong?
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1164
Location: /run/user/1000

PostPosted: Thu Jul 19, 2018 9:33 am    Post subject: Reply with quote

So, what happens if verifying fails ?
Process stops right there with clear notification about it ?
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1179

PostPosted: Thu Jul 19, 2018 9:53 am    Post subject: Reply with quote

Not really, repeated atempts go on indefinitely to verify the key. As far as I recall until you manually stop it from doing so.
Back to top
View user's profile Send private message
Aiken
Apprentice
Apprentice


Joined: 22 Jan 2003
Posts: 239
Location: Toowoomba/Australia

PostPosted: Thu Jul 19, 2018 10:51 pm    Post subject: Reply with quote

kajzer wrote:
So, what happens if verifying fails ?
Process stops right there with clear notification about it ?


On the one machine I have it turned on my experiences are usually somewhere over a minute for the refresh keys and the --sync I just finished ended with

Code:

 * Verifying /usr/portage ...!!! Manifest verification failed:
Manifest mismatch for sys-apps/Manifest.gz
  __size__: expected: 49672, have: 49671
q: Updating ebuild cache in /usr/portage ...
q: Finished 35511 entries in 0.161402 seconds

Action: sync for repo: gentoo, returned code = 1


My cron job sends me an email when the return code is 1. It is a rare day I do not get such an email. With this example running emerge -avuDN world portage wants to update sys-apps/file so such a comment at the end of the --sync does not stop it from using the part of the tree with that message. Have on off site machine with the verify still turned on and it is common for it to also complain about a mismatch somewhere. Probably no surprise all the machines with verify turned off never complain.
_________________
Beware the grue.
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1179

PostPosted: Fri Jul 20, 2018 6:58 am    Post subject: Reply with quote

It would appear that the package gemato has 2 bugs filed, one of them dealing with the "deadlock" but for some reason it is a much earlier version of the package like in April of this year.

But since there is was a patch provided there I think it would make sense to attempt this first, as it was provided by the maintainer but I would imagine he has a lot going on and can't provide solutions in such a fast way to the general community if working on so many fronts simultaneously for example different architectures or whatever.

For now I also have disabled the feature.

But in case it could help then this the terminal output I get when stopping the job after waiting a good 10 minutes before using Ctrl+c to be stopping job:

Code:
  File "/usr/lib/python-exec/python2.7/gemato", line 11, in <module>
    load_entry_point('gemato==13.1', 'console_scripts', 'gemato')()
  File "/usr/lib64/python2.7/site-packages/gemato/cli.py", line 549, in setuptools_main
    sys.exit(main(argv))
  File "/usr/lib64/python2.7/site-packages/gemato/cli.py", line 535, in main
    return vals.cmd()
  File "/usr/lib64/python2.7/site-packages/gemato/cli.py", line 231, in __call__
    ret &= m.assert_directory_verifies(relpath, **self.kwargs)
  File "/usr/lib64/python2.7/site-packages/gemato/recursiveloader.py", line 654, in assert_directory_verifies
    chunksize=64))
  File "/usr/lib64/python2.7/site-packages/gemato/util.py", line 36, in imap_unordered
    return self.map(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/gemato/util.py", line 29, in map
    return map(func, it)
  File "/usr/lib64/python2.7/site-packages/gemato/recursiveloader.py", line 135, in __call__
    fpath, fe)
  File "/usr/lib64/python2.7/site-packages/gemato/recursiveloader.py", line 95, in _verify_one_file
    last_mtime=self.last_mtime)
  File "/usr/lib64/python2.7/site-packages/gemato/verify.py", line 169, in verify_path
    exists = next(g)
  File "/usr/lib64/python2.7/site-packages/gemato/verify.py", line 40, in get_file_metadata
    fd = os.open(path, os.O_RDONLY|os.O_NONBLOCK)
KeyboardInterrupt
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1164
Location: /run/user/1000

PostPosted: Fri Jul 27, 2018 6:15 pm    Post subject: Reply with quote

Well, it just happened to me, no repeated attempts, process just stopped :

Code:
sent 147.24K bytes  received 14.12M bytes  133.97K bytes/sec
total size is 215.62M  speedup is 15.11
 * Manifest timestamp: 2018-07-27 17:38:34 UTC
 * Valid OpenPGP signature found:
 * - primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D
 * - subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
 * - timestamp: 2018-07-27 17:38:34 UTC
 * Verifying /usr/portage/.tmp-unverified-download-quarantine ...!!! Manifest verification failed:
Manifest mismatch for app-portage/elt-patches/elt-patches-20170422.ebuild
  SHA512: expected: 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731`78236c02b4bb4adc660d31979ba6e06d, have: 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731a78236c02b4bb4adc660d31979ba6e06d
q: Updating ebuild cache for /usr/portage ...
q: Finished 35362 entries in 8.348029 seconds
Action: sync for repo: gentoo, returned code = 1
 * emerge --sync failed
 * Time statistics:
   347 seconds for syncing
   348 seconds total
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Fri Jul 27, 2018 6:59 pm    Post subject: Reply with quote

kajzer wrote:
Code:
  SHA512:
expected: 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731`78236c02b4bb4adc660d31979ba6e06d
    have: 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731a78236c02b4bb4adc660d31979ba6e06d

Looks like one bit is toggled. Maybe a RAM problem?
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1164
Location: /run/user/1000

PostPosted: Fri Jul 27, 2018 7:25 pm    Post subject: Reply with quote

mv wrote:

Looks like one bit is toggled. Maybe a RAM problem?


Problems no, if you mean low RAM then maybe, I have 2GB
Anyway, 5 minutes later I did another try and it went fine.
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3106

PostPosted: Fri Jul 27, 2018 7:43 pm    Post subject: Reply with quote

Well spotted, mv, the first hash is invalid.
Code:

b669731`78236c02b
b669731a78236c02b

Quote:
kajzer
, you better run an extensive memtest there. It seems that your machine actually made a mistake.
Quote:
Anyway, 5 minutes later I did another try and it went fine.
Yeah, you don't expect random errors to pop up very often, do you? :)
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1164
Location: /run/user/1000

PostPosted: Fri Jul 27, 2018 7:53 pm    Post subject: Reply with quote

I'm not saying my machine didn't made a mistake, I just reported what happened.
It looks like it did made a mistake, however I'm not having other problems with this machine.
I'm sure memory is fine, if verifying needs a lot of RAM and it's possible that it would fail on low free RAM then that could be it.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Goto page Previous  1, 2, 3, 4  Next
Page 3 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum