RPC failed; HTTP 504 curl 22 while syncing via git

Message

Althorion · Post by **Althorion** » Fri Oct 16, 2020 1:06 am

When I try to sync Gentoo’s main repo via git, I keep getting the following error:

# emerge --verbose --sync gentoo
/usr/bin/git fetch origin
error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504
fatal: the remote end hung up unexpectedly
!!! git fetch error in /var/db/repos/gentoo

Action: sync for repo: gentoo, returned code = 128

But I don’t think this is, in fact, the network error—I have tried two different URIs (https://anongit.gentoo.org/git/repo/sync/gentoo.git and https://github.com/gentoo-mirror/gentoo.git) and waited a week for it to resolve on its own, but no dice. I can pull the repo manually, but not via Portage.

My /etc/portage/repos.conf/gentoo.conf:

Code: Select all

[DEFAULT]
main-repo = gentoo

[gentoo]
location = /var/db/repos/gentoo
sync-type = git
sync-uri = https://anongit.gentoo.org/git/repo/sync/gentoo.git
# sync-uri = https://gitweb.gentoo.org/repo/gentoo.git

Post by **Banana** » Fri Oct 16, 2020 5:54 am

I do not get any content while accessing: https://anongit.gentoo.org/git/repo/sync/gentoo.git
Can you use a non http(s) URI?

Post by **Ionen** » Fri Oct 16, 2020 6:06 am

Could try with `GIT_TRACE=1 GIT_CURL_VERBOSE=1 emerge --sync gentoo`, may give some more clues. Although 504 is normally for gateway timeout, I sync with that anongit link myself and it's fine on my end (well, my mirror of it is updating fine anyway.. my setup is a bit different).

Things being different under portage while direct git command works may not be entirely surprising (given all the features/restrictions), but I'd expect another error

Edit:
Could try to remove the tree too (e.g. let it clone again), if you've been messing with it maybe it's confusing the server and failing with the commands portage use.

Banana wrote:I do not get any content while accessing: https://anongit.gentoo.org/git/repo/sync/gentoo.git

That's because it can only be access with git, web-only version is at https://gitweb.gentoo.org/repo/sync/gentoo.git/

Althorion · Post by **Althorion** » Fri Oct 16, 2020 8:43 am

First sync (i.e., `rm -rf /var/db/repos/gentoo && emerge --sync gentoo`) goes through on both HTTP and HTTPS and both using gentoo.org and GitHub. But any subsequent pull fails like this:

After changing to non-HTTPS version:

Code: Select all

GIT_TRACE=1 GIT_CURL_VERBOSE=1 emerge --sync gentoo
>>> Syncing repository 'gentoo' into '/var/db/repos/gentoo'...
10:41:10.252990 git.c:444               trace: built-in: git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}'
/usr/bin/git fetch origin
10:41:10.255819 git.c:444               trace: built-in: git rev-list --max-count=1 HEAD
10:41:10.261411 git.c:444               trace: built-in: git fetch origin
10:41:10.275364 run-command.c:663       trace: run_command: GIT_DIR=.git git-remote-http origin http://anongit.gentoo.org/git/repo/sync/gentoo.git
10:41:10.278215 http.c:756              == Info: Couldn't find host anongit.gentoo.org in the .netrc file; using defaults
10:41:10.402920 http.c:756              == Info:   Trying 88.198.51.10:80...
10:41:10.441586 http.c:756              == Info: Connected to anongit.gentoo.org (88.198.51.10) port 80 (#0)
10:41:10.441642 http.c:703              => Send header, 0000000195 bytes (0x000000c3)
10:41:10.441648 http.c:715              => Send header: GET /git/repo/sync/gentoo.git/info/refs?service=git-upload-pack HTTP/1.1
10:41:10.441651 http.c:715              => Send header: Host: anongit.gentoo.org
10:41:10.441654 http.c:715              => Send header: User-Agent: git/2.28.0
10:41:10.441657 http.c:715              => Send header: Accept: */*
10:41:10.441659 http.c:715              => Send header: Accept-Encoding: deflate, gzip, zstd
10:41:10.441662 http.c:715              => Send header: Pragma: no-cache
10:41:10.441663 http.c:715              => Send header:
10:41:10.492495 http.c:756              == Info: Mark bundle as not supporting multiuse
10:41:10.492521 http.c:703              <= Recv header, 0000000017 bytes (0x00000011)
10:41:10.492526 http.c:715              <= Recv header: HTTP/1.1 200 OK
10:41:10.492530 http.c:703              <= Recv header, 0000000015 bytes (0x0000000f)
10:41:10.492533 http.c:715              <= Recv header: Server: nginx
10:41:10.492536 http.c:703              <= Recv header, 0000000037 bytes (0x00000025)
10:41:10.492538 http.c:715              <= Recv header: Date: Fri, 16 Oct 2020 08:41:10 GMT
10:41:10.492542 http.c:703              <= Recv header, 0000000059 bytes (0x0000003b)
10:41:10.492545 http.c:715              <= Recv header: Content-Type: application/x-git-upload-pack-advertisement
10:41:10.492550 http.c:703              <= Recv header, 0000000028 bytes (0x0000001c)
10:41:10.492553 http.c:715              <= Recv header: Transfer-Encoding: chunked
10:41:10.492556 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
10:41:10.492559 http.c:715              <= Recv header: Connection: keep-alive
10:41:10.492562 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
10:41:10.492564 http.c:715              <= Recv header: Keep-Alive: timeout=20
10:41:10.492567 http.c:703              <= Recv header, 0000000040 bytes (0x00000028)
10:41:10.492570 http.c:715              <= Recv header: Expires: Fri, 16 Oct 2020 08:56:10 GMT
10:41:10.492573 http.c:703              <= Recv header, 0000000018 bytes (0x00000012)
10:41:10.492576 http.c:715              <= Recv header: Pragma: no-cache
10:41:10.492579 http.c:703              <= Recv header, 0000000028 bytes (0x0000001c)
10:41:10.492582 http.c:715              <= Recv header: Cache-Control: max-age=900
10:41:10.492585 http.c:703              <= Recv header, 0000000155 bytes (0x0000009b)
10:41:10.492588 http.c:715              <= Recv header: Content-Security-Policy: default-src 'self' https://assets.gentoo.org; img-src *; media-src *; style-src 'self' https://assets.gentoo.org 'unsafe-inline'
10:41:10.492592 http.c:703              <= Recv header, 0000000002 bytes (0x00000002)
10:41:10.492595 http.c:715              <= Recv header:
10:41:10.492609 http.c:756              == Info: Connection #0 to host anongit.gentoo.org left intact
10:41:10.492897 run-command.c:663       trace: run_command: git fetch-pack --stateless-rpc --stdin --lock-pack --include-tag --thin http://anongit.gentoo.org/git/repo/sync/gentoo.git/
10:41:10.494050 git.c:444               trace: built-in: git fetch-pack --stateless-rpc --stdin --lock-pack --include-tag --thin http://anongit.gentoo.org/git/repo/sync/gentoo.git/
10:41:10.494403 http.c:756              == Info: Couldn't find host anongit.gentoo.org in the .netrc file; using defaults
10:41:10.494420 http.c:756              == Info: Found bundle for host anongit.gentoo.org: 0x55906cd4ba50 [serially]
10:41:10.494424 http.c:756              == Info: Can not multiplex, even if we wanted to!
10:41:10.494428 http.c:756              == Info: Re-using existing connection! (#0) with host anongit.gentoo.org
10:41:10.494434 http.c:756              == Info: Connected to anongit.gentoo.org (88.198.51.10) port 80 (#0)
10:41:10.494471 http.c:703              => Send header, 0000000267 bytes (0x0000010b)
10:41:10.494477 http.c:715              => Send header: POST /git/repo/sync/gentoo.git/git-upload-pack HTTP/1.1
10:41:10.494479 http.c:715              => Send header: Host: anongit.gentoo.org
10:41:10.494482 http.c:715              => Send header: User-Agent: git/2.28.0
10:41:10.494484 http.c:715              => Send header: Accept-Encoding: deflate, gzip, zstd
10:41:10.494487 http.c:715              => Send header: Content-Type: application/x-git-upload-pack-request
10:41:10.494489 http.c:715              => Send header: Accept: application/x-git-upload-pack-result
10:41:10.494492 http.c:715              => Send header: Content-Length: 279
10:41:10.494494 http.c:715              => Send header:
10:41:10.494498 http.c:756              == Info: upload completely sent off: 279 out of 279 bytes
10:42:10.542544 http.c:756              == Info: Mark bundle as not supporting multiuse
10:42:10.542568 http.c:703              <= Recv header, 0000000031 bytes (0x0000001f)
10:42:10.542573 http.c:715              <= Recv header: HTTP/1.1 504 Gateway Time-out
10:42:10.542578 http.c:703              <= Recv header, 0000000015 bytes (0x0000000f)
10:42:10.542582 http.c:715              <= Recv header: Server: nginx
10:42:10.542586 http.c:703              <= Recv header, 0000000037 bytes (0x00000025)
10:42:10.542589 http.c:715              <= Recv header: Date: Fri, 16 Oct 2020 08:42:10 GMT
10:42:10.542593 http.c:703              <= Recv header, 0000000025 bytes (0x00000019)
10:42:10.542595 http.c:715              <= Recv header: Content-Type: text/html
10:42:10.542598 http.c:703              <= Recv header, 0000000022 bytes (0x00000016)
10:42:10.542600 http.c:715              <= Recv header: Content-Length: 2283
10:42:10.542603 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
10:42:10.542605 http.c:715              <= Recv header: Connection: keep-alive
10:42:10.542607 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
10:42:10.542609 http.c:715              <= Recv header: Keep-Alive: timeout=20
10:42:10.542611 http.c:703              <= Recv header, 0000000022 bytes (0x00000016)
10:42:10.542613 http.c:715              <= Recv header: ETag: "5e125fbc-8eb"
10:42:10.542616 http.c:703              <= Recv header, 0000000002 bytes (0x00000002)
10:42:10.542618 http.c:715              <= Recv header:
10:42:10.594549 http.c:756              == Info: Connection #0 to host anongit.gentoo.org left intact
error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504
fatal: the remote end hung up unexpectedly
!!! git fetch error in /var/db/repos/gentoo

Action: sync for repo: gentoo, returned code = 128

EDIT: I’ve also tried a different Internet connection (cellular, via USB)—fails with the same error.

Post by **Banana** » Mon Oct 19, 2020 7:30 am

@Ionen thanks for the hint.

And I can also clone http://anongit.gentoo.org/git/repo/sync/gentoo.git without any problems.

gtbX · Post by **gtbX** » Wed Jan 27, 2021 6:13 am

I was having this issue too. The initial sync with https://anongit.gentoo.org/git/repo/sync/gentoo.git would work fine, but any sync after that would 504 (timeout).

On a hunch, I manually deleted /var/db/repos/gentoo and cloned it with `git clone`. I've now sunc it both manually with `git pull` and with `emaint sync` and it's worked both times.

I think by default when using git, the initial clone is a "shallow" clone (i.e. repo history is not downloaded). That's fine for a one-time download, but apparently it makes later pulls cpu-intensive on the server side. At least until your local copy has built up enough history maybe?
IIUC, `git -C /var/db/repos/gentoo fetch --unshallow` should have the same effect as deleting and re-cloning.

What gave me the idea was another project, "homebrew" for macOS, that uses git internally for it's package repo. They recently required uses to "deepen" local copies of the repo to speed up syncing. (https://github.com/Homebrew/brew/pull/9383)

gtbX · Post by **gtbX** » Sun Feb 07, 2021 5:54 am

I tested this again but this time specifying "clone-depth = 10" in my repos.conf/gentoo. This produced a configuration that both cloned and synced fairly quickly (obviously, ymmv)

Code: Select all

[DEFAULT]
main-repo = gentoo

[gentoo]
location = /var/db/repos/gentoo
sync-type = git
sync-uri = https://anongit.gentoo.org/git/repo/sync/gentoo.git
clone-depth = 10
sync-git-verify-commit-signature = yes
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc

Althorion · Post by **Althorion** » Sun Feb 07, 2021 5:56 pm

Thank you very much for your investigation. With that change everything seems to be working just fine—at least for now.

papu · Post by **papu** » Fri Apr 23, 2021 7:07 pm

i have same problem , it seems when there are new update of any package on git , it gets
error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504

because if i sync git immediately severalt times it works and always before i delete /var/db/repos/gentoo

well really i don't know

g0rg0n · Post by **g0rg0n** » Tue Jul 13, 2021 2:38 am

gtbX wrote:I tested this again but this time specifying "clone-depth = 10" in my repos.conf/gentoo. This produced a configuration that both cloned and synced fairly quickly (obviously, ymmv)
Code: Select all
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /var/db/repos/gentoo
sync-type = git
sync-uri = https://anongit.gentoo.org/git/repo/sync/gentoo.git
clone-depth = 10
sync-git-verify-commit-signature = yes
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc

I have the same issue but the issue persists with clone-depth = 10

Code: Select all

coruscant ~ # GIT_TRACE=1 GIT_CURL_VERBOSE=1 emerge --sync gentoo
>>> Syncing repository 'gentoo' into '/var/db/repos/gentoo'...
12:34:33.935756 git.c:447               trace: built-in: git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}'
/usr/bin/git fetch origin
12:34:33.941342 git.c:447               trace: built-in: git rev-list --max-count=1 HEAD
12:34:33.951175 git.c:447               trace: built-in: git fetch origin
12:34:33.984385 run-command.c:667       trace: run_command: GIT_DIR=.git git remote-https origin https://anongit.gentoo.org/git/repo/sync/gentoo.git
12:34:33.988881 git.c:733               trace: exec: git-remote-https origin https://anongit.gentoo.org/git/repo/sync/gentoo.git
12:34:33.988971 run-command.c:667       trace: run_command: git-remote-https origin https://anongit.gentoo.org/git/repo/sync/gentoo.git
12:34:33.994332 http.c:756              == Info: Couldn't find host anongit.gentoo.org in the (nil) file; using defaults
12:34:34.012216 http.c:756              == Info:   Trying 148.251.78.52:443...
12:34:34.213244 http.c:756              == Info:   Trying 2a01:4f8:202:4333::2:443...
12:34:34.213359 http.c:756              == Info: Immediate connect fail for 2a01:4f8:202:4333::2: Network is unreachable
12:34:34.345196 http.c:756              == Info: Connected to anongit.gentoo.org (148.251.78.52) port 443 (#0)
12:34:34.346188 http.c:756              == Info: ALPN, offering h2
12:34:34.346218 http.c:756              == Info: ALPN, offering http/1.1
12:34:34.365433 http.c:756              == Info: successfully set certificate verify locations:
12:34:34.365478 http.c:756              == Info:  CAfile: /etc/ssl/certs/ca-certificates.crt
12:34:34.365496 http.c:756              == Info:  CApath: /etc/ssl/certs
12:34:34.366215 http.c:756              == Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
12:34:34.714314 http.c:756              == Info: TLSv1.3 (IN), TLS handshake, Server hello (2):
12:34:34.724600 http.c:756              == Info: TLSv1.2 (IN), TLS handshake, Certificate (11):
12:34:34.726296 http.c:756              == Info: TLSv1.2 (IN), TLS handshake, Server key exchange (12):
12:34:34.727142 http.c:756              == Info: TLSv1.2 (IN), TLS handshake, Server finished (14):
12:34:34.727726 http.c:756              == Info: TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
12:34:34.727809 http.c:756              == Info: TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
12:34:34.728021 http.c:756              == Info: TLSv1.2 (OUT), TLS handshake, Finished (20):
12:34:35.063158 http.c:756              == Info: TLSv1.2 (IN), TLS handshake, Finished (20):
12:34:35.063285 http.c:756              == Info: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
12:34:35.063312 http.c:756              == Info: ALPN, server accepted to use http/1.1
12:34:35.063350 http.c:756              == Info: Server certificate:
12:34:35.063396 http.c:756              == Info:  subject: CN=anon.gentoo.org
12:34:35.063429 http.c:756              == Info:  start date: Jun 23 01:30:15 2021 GMT
12:34:35.063455 http.c:756              == Info:  expire date: Sep 21 01:30:14 2021 GMT
12:34:35.063513 http.c:756              == Info:  subjectAltName: host "anongit.gentoo.org" matched cert's "anongit.gentoo.org"
12:34:35.063568 http.c:756              == Info:  issuer: C=US; O=Let's Encrypt; CN=R3
12:34:35.063587 http.c:756              == Info:  SSL certificate verify ok.
12:34:35.063706 http.c:703              => Send header, 0000000214 bytes (0x000000d6)
12:34:35.063729 http.c:715              => Send header: GET /git/repo/sync/gentoo.git/info/refs?service=git-upload-pack HTTP/1.1
12:34:35.063740 http.c:715              => Send header: Host: anongit.gentoo.org
12:34:35.063748 http.c:715              => Send header: User-Agent: git/2.31.1
12:34:35.063757 http.c:715              => Send header: Accept: */*
12:34:35.063766 http.c:715              => Send header: Accept-Encoding: deflate, gzip
12:34:35.063779 http.c:715              => Send header: Pragma: no-cache
12:34:35.063792 http.c:715              => Send header: Git-Protocol: version=2
12:34:35.063802 http.c:715              => Send header:
12:34:35.405622 http.c:756              == Info: Mark bundle as not supporting multiuse
12:34:35.405713 http.c:703              <= Recv header, 0000000017 bytes (0x00000011)
12:34:35.405736 http.c:715              <= Recv header: HTTP/1.1 200 OK
12:34:35.405755 http.c:703              <= Recv header, 0000000015 bytes (0x0000000f)
12:34:35.405769 http.c:715              <= Recv header: Server: nginx
12:34:35.405784 http.c:703              <= Recv header, 0000000037 bytes (0x00000025)
12:34:35.405797 http.c:715              <= Recv header: Date: Tue, 13 Jul 2021 02:34:35 GMT
12:34:35.405816 http.c:703              <= Recv header, 0000000059 bytes (0x0000003b)
12:34:35.405828 http.c:715              <= Recv header: Content-Type: application/x-git-upload-pack-advertisement
12:34:35.405846 http.c:703              <= Recv header, 0000000028 bytes (0x0000001c)
12:34:35.405860 http.c:715              <= Recv header: Transfer-Encoding: chunked
12:34:35.405878 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
12:34:35.405891 http.c:715              <= Recv header: Connection: keep-alive
12:34:35.405906 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
12:34:35.405919 http.c:715              <= Recv header: Keep-Alive: timeout=20
12:34:35.405930 http.c:703              <= Recv header, 0000000040 bytes (0x00000028)
12:34:35.405940 http.c:715              <= Recv header: Expires: Tue, 13 Jul 2021 02:49:35 GMT
12:34:35.405954 http.c:703              <= Recv header, 0000000018 bytes (0x00000012)
12:34:35.405967 http.c:715              <= Recv header: Pragma: no-cache
12:34:35.405982 http.c:703              <= Recv header, 0000000028 bytes (0x0000001c)
12:34:35.405995 http.c:715              <= Recv header: Cache-Control: max-age=900
12:34:35.406012 http.c:703              <= Recv header, 0000000155 bytes (0x0000009b)
12:34:35.406027 http.c:715              <= Recv header: Content-Security-Policy: default-src 'self' https://assets.gentoo.org; img-src *; media-src *; style-src 'self' https://assets.gentoo.org 'unsafe-inline'
12:34:35.406045 http.c:703              <= Recv header, 0000000002 bytes (0x00000002)
12:34:35.406055 http.c:715              <= Recv header:
12:34:35.406111 http.c:756              == Info: Connection #0 to host anongit.gentoo.org left intact
12:34:35.406913 run-command.c:667       trace: run_command: git fetch-pack --stateless-rpc --stdin --lock-pack --include-tag --thin https://anongit.gentoo.org/git/repo/sync/gentoo.git/
12:34:35.409933 git.c:447               trace: built-in: git fetch-pack --stateless-rpc --stdin --lock-pack --include-tag --thin https://anongit.gentoo.org/git/repo/sync/gentoo.git/
12:34:35.411023 http.c:756              == Info: Couldn't find host anongit.gentoo.org in the (nil) file; using defaults
12:34:35.411156 http.c:756              == Info: Found bundle for host anongit.gentoo.org: 0x555cbdab4df0 [serially]
12:34:35.411172 http.c:756              == Info: Can not multiplex, even if we wanted to!
12:34:35.411375 http.c:756              == Info: Re-using existing connection! (#0) with host anongit.gentoo.org
12:34:35.411399 http.c:756              == Info: Connected to anongit.gentoo.org (148.251.78.52) port 443 (#0)
12:34:35.411615 http.c:703              => Send header, 0000000261 bytes (0x00000105)
12:34:35.411683 http.c:715              => Send header: POST /git/repo/sync/gentoo.git/git-upload-pack HTTP/1.1
12:34:35.411697 http.c:715              => Send header: Host: anongit.gentoo.org
12:34:35.411753 http.c:715              => Send header: User-Agent: git/2.31.1
12:34:35.411810 http.c:715              => Send header: Accept-Encoding: deflate, gzip
12:34:35.411821 http.c:715              => Send header: Content-Type: application/x-git-upload-pack-request
12:34:35.411830 http.c:715              => Send header: Accept: application/x-git-upload-pack-result
12:34:35.411839 http.c:715              => Send header: Content-Length: 279
12:34:35.411848 http.c:715              => Send header:
12:35:35.757917 http.c:756              == Info: Mark bundle as not supporting multiuse
12:35:35.757976 http.c:703              <= Recv header, 0000000031 bytes (0x0000001f)
12:35:35.758024 http.c:715              <= Recv header: HTTP/1.1 504 Gateway Time-out
12:35:35.758058 http.c:703              <= Recv header, 0000000015 bytes (0x0000000f)
12:35:35.758088 http.c:715              <= Recv header: Server: nginx
12:35:35.758119 http.c:703              <= Recv header, 0000000037 bytes (0x00000025)
12:35:35.758148 http.c:715              <= Recv header: Date: Tue, 13 Jul 2021 02:35:35 GMT
12:35:35.758190 http.c:703              <= Recv header, 0000000025 bytes (0x00000019)
12:35:35.758220 http.c:715              <= Recv header: Content-Type: text/html
12:35:35.758253 http.c:703              <= Recv header, 0000000022 bytes (0x00000016)
12:35:35.758283 http.c:715              <= Recv header: Content-Length: 2283
12:35:35.758328 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
12:35:35.758358 http.c:715              <= Recv header: Connection: keep-alive
12:35:35.758393 http.c:703              <= Recv header, 0000000024 bytes (0x00000018)
12:35:35.758452 http.c:715              <= Recv header: Keep-Alive: timeout=20
12:35:35.758470 http.c:703              <= Recv header, 0000000022 bytes (0x00000016)
12:35:35.758483 http.c:715              <= Recv header: ETag: "5e126410-8eb"
12:35:35.758496 http.c:703              <= Recv header, 0000000002 bytes (0x00000002)
12:35:35.758506 http.c:715              <= Recv header:
12:35:35.758555 http.c:756              == Info: Connection #0 to host anongit.gentoo.org left intact
error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504
fatal: the remote end hung up unexpectedly
!!! git fetch error in /var/db/repos/gentoo

Action: sync for repo: gentoo, returned code = 128


coruscant ~ #

any suggestions? maybe I should switch back to rsync.

edit: Strangly,

Code: Select all

rm -rf /var/db/repos/gentoo && emerge --sync gentoo

works

papu · Post by **papu** » Wed Jul 14, 2021 2:22 pm

Code: Select all

[DEFAULT]
main-repo = gentoo

[gentoo]
location = /var/db/repos/gentoo
auto-sync = yes
sync-depth = 1
sync-type = git
sync-uri = https://github.com/gentoo-mirror/gentoo.git
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
sync-openpgp-keyserver = hkps://keys.gentoo.org
sync-openpgp-key-refresh-retry-count = 40
sync-openpgp-key-refresh-retry-overall-timeout = 1200
sync-openpgp-key-refresh-retry-delay-exp-base = 2
sync-openpgp-key-refresh-retry-delay-max = 60
sync-openpgp-key-refresh-retry-delay-mult = 4
sync-webrsync-verify-signature = yes

this works forme i am using for 1 month ago

g0rg0n · Post by **g0rg0n** » Thu Jul 15, 2021 1:27 am

hmmm I did not change anything and it's working now...

Code: Select all

coruscant ~ # emerge --sync gentoo
>>> Syncing repository 'gentoo' into '/var/db/repos/gentoo'...
/usr/bin/git fetch origin
remote: Enumerating objects: 3053202, done.
remote: Counting objects: 100% (3053145/3053145), done.
remote: Compressing objects: 100% (985677/985677), done.
Receiving objects:   2% (77059/3020550), 52.55 MiB | 1.39 MiB/s

Code: Select all

obiwan at coruscant in ~
$ cat /etc/portage/repos.conf/gentoo.conf
[DEFAULT]
main-repo = gentoo

[gentoo]
location = /var/db/repos/gentoo
#sync-type = rsync
#sync-uri = rsync://rsync.au.gentoo.org/gentoo-portage

sync-type = git
#sync-uri = http://anongit.gentoo.org/git/repo/sync/gentoo.git
sync-uri = https://github.com/gentoo-mirror/gentoo.git
#clone-depth = 5
#sync-git-verify-commit-signature = yes
auto-sync = yes
sync-rsync-verify-jobs = 1
sync-rsync-verify-metamanifest = yes
sync-rsync-verify-max-age = 24
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
sync-openpgp-keyserver = hkps://keys.gentoo.org
sync-openpgp-key-refresh-retry-count = 40
sync-openpgp-key-refresh-retry-overall-timeout = 1200
sync-openpgp-key-refresh-retry-delay-exp-base = 2
sync-openpgp-key-refresh-retry-delay-max = 60
sync-openpgp-key-refresh-retry-delay-mult = 4
sync-webrsync-verify-signature = yes

obiwan at coruscant in ~
$

cyisfor · Post by **cyisfor** » Tue Oct 05, 2021 5:40 pm

I imagine if you request clone-depth=9001, and a bajillion others are going with more server intensive operations, it’ll 504 timeout for all of you. That’s probably why it’s not working, even with clone-depth=10. Just keep trying every now and again I guess?

But maybe not. Go to the /var/db/repos/gentoo (or wherever your repository is) directory, and run `git fetch --depth=10 origin` yourself. That will get around if portage is ignoring “clone-depth” for some reason. You’ll also probably have to run `git reset --hard origin/master` since deeper clones have unrelated histories to shallower ones, which git tries to merge and fails.

After that, the sync should succeed, and then portage should upgrade itself to a version that supports `clone-depth`. Or something. I dunno what’s going on there. Portage just tells me it’s running “git fetch origin” in the UI, not saying the --depth it’s specifying.

frony0 · Post by **frony0** » Sun Jan 30, 2022 9:22 pm

I was seeing this too. Resolved by switching to the github mirror for the unshallow:

Code: Select all

git -C /var/db/repos/gentoo fetch https://github.com/gentoo-mirror/gentoo.git --unshallow

duxsco · Post by **duxsco** » Thu Mar 31, 2022 9:15 pm

I explained the cause for this issue in my comment at:
https://bugs.gentoo.org/824782#c17

The PR to fix this is linked at the end.