| View previous topic :: View next topic |
| Author |
Message |
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
 Posted: Tue Sep 18, 2018 12:10 pm Post subject: WARNING: DO NOT SYNC! (it emptied my portage tree) |
 |
|
I just sync'ed my portage tree, but something failed in a bad way and now my portage tree has been emptied, apparently.
| Code: | directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/x11-libs" [42/1821]
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/x11-misc"
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/x11-plugins"
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/x11-terms"
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/x11-themes"
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/x11-wm"
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/xfce-base"
directory has vanished: "/usr/portage/.tmp-unverified-download-quarantine/xfce-extra"
file has vanished: "/usr/portage/.tmp-unverified-download-quarantine/app-admin/Manifest.gz"
Number of files: 1,179 (reg: 691, dir: 488)
Number of created files: 0
Number of deleted files: 162,764 (reg: 135,255, dir: 27,509)
Number of regular files transferred: 3
Total file size: 977.91K bytes
Total transferred file size: 65.33K bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 32.20K
Total bytes received: 12.82M
sent 32.20K bytes received 12.82M bytes 2.86M bytes/sec
total size is 977.91K speedup is 0.08
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1189) [sender=3.1.3]
q: Updating ebuild cache in /usr/portage ...
q: Finished 227 entries in 0.169312 seconds
Action: sync for repo: gentoo, returned code = 24
* An update to portage is available. It is _highly_ recommended
* that you update portage now, before any other packages are updated.
* To update portage, run 'emerge --oneshot portage' now.
* emerge --sync failed
* Time statistics:
662 seconds for syncing
664 seconds total |
In fact, I knew I had to emerge @preserved-rebuild (for rust), but decided to sync my tree first. So this now gives
| Code: | # emerge -qva @preserved-rebuild
!!! /etc/portage/make.profile is not a symlink and will probably prevent most merges.
!!! It should point into a profile within /usr/portage/profiles/
!!! (You can safely ignore this message when syncing. It's harmless.)
!!! Your current profile is invalid. If you have just changed your profile
!!! configuration, you should revert back to the previous configuration.
!!! Allowed actions are limited to --help, --info, --search, --sync, and
!!! --version. |
And so I now have no profile file anymore!
A question mark is also listed after Package Database on https://infra-status.gentoo.org/.
What is going on?
I guess I best wait until this is fixed and then try and re-sync. Please let me know when that is once you find out. Thanks. |
|
| Back to top |
|
 |
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 12:33 pm Post subject: |
|
|
Tried. The same problem as before. (I sync using the classical rsync method.) |
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
| Posted: Tue Sep 18, 2018 12:38 pm Post subject: |
|
|
Could be a bad rsync server maybe? https://www.gentoo.org/support/rsync-mirrors/
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Tue Sep 18, 2018 12:41 pm Post subject: |
|
|
I had no trouble as of 3 AM this morning.
| Code: | | GENTOO_MIRRORS="rsync://rsync.gtlib.gatech.edu/gentoo ftp://ftp.ussg.iu.edu/pub/linux/gentoo http://gentoo.osuosl.org/ http://ftp.ucsb.edu/pub/mirrors/linux/gentoo/ rsync://gentoo.cs.uni.edu/gentoo-distfiles http://mirror.usu.edu/mirrors/gentoo/" |
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 12:42 pm Post subject: |
|
|
Then it is more than just one mirror:
| Quote: | 1537270442: Started emerge on: sep 18, 2018 13:34:01
1537270442: *** emerge --misspell-suggestions=n --with-bdeps=y --sync
1537270442: === sync
1537270442: >>> Syncing repository 'gentoo' into '/usr/portage'...
1537270446: >>> Starting rsync with rsync://193.136.164.6/gentoo-portage
1537270714: *** terminating.
1537270818: Started emerge on: sep 18, 2018 13:40:18
1537270818: *** emerge --misspell-suggestions=n --with-bdeps=y --sync
1537270818: === sync
1537270818: >>> Syncing repository 'gentoo' into '/usr/portage'...
1537270823: >>> Starting rsync with rsync://178.210.128.3/gentoo-portage
1537271480: *** terminating.
1537273891: Started emerge on: sep 18, 2018 14:31:31
1537273891: *** emerge --misspell-suggestions=n --with-bdeps=y --sync
1537273891: === sync
1537273891: >>> Syncing repository 'gentoo' into '/usr/portage'...
1537273896: >>> Starting rsync with rsync://92.60.51.128/gentoo-portage
1537273946: *** terminating. |
|
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5934
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Tue Sep 18, 2018 1:30 pm Post subject: |
|
|
I doubt it's a whole lot of different servers, more likely something on your end/your system.
| Code: | 1537254002: Started emerge on: Sep 18, 2018 02:00:01
1537254002: *** emerge --with-bdeps=y --quiet-build=y --autounmask=n --sync
1537254002: === sync
1537254002: >>> Syncing repository 'gentoo' into '/usr/portage'...
1537254015: >>> Starting rsync with rsync://91.186.30.235/gentoo-portage
1537254094: === Sync completed for gentoo |
That was this morning.
and it corresponds to rsync://rsync.gtlib.gatech.edu/gentoo
Looking at the time stamp they seem to be active for ~50 seconds which is a little short, mine from this morning was 1 min and 20 seconds.
Edit to add: what does "cat /etc/portage/repos.conf/gentoo.conf" return
ETA2: is /usr/portage on the root filesystem, or a mounted filesystem?
What have you changed on your system lately, fstab, etc?
If you do "touch /usr/portage/test.file" is there a file called "test.file" in /usr/portage?
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 1:41 pm Post subject: |
|
|
No. |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 1:49 pm Post subject: |
|
|
I've changed my rsync mirror to the US one from the European one and am now syncing using the gatech mirror (goes very slowly!). Will report back once it's done.
| Anon-E-moose wrote: | | what does "cat /etc/portage/repos.conf/gentoo.conf" return |
| Code: | $ cat /etc/portage/repos.conf/gentoo.conf
[DEFAULT]
main-repo = gentoo
[gentoo]
location = /usr/portage
sync-type = rsync
sync-uri = rsync://rsync.us.gentoo.org/gentoo-portage
#sync-uri = rsync://rsync.europe.gentoo.org/gentoo-portage |
| Anon-E-moose wrote: | | is /usr/portage on the root filesystem, or a mounted filesystem? |
root file system
| Anon-E-moose wrote: | | What have you changed on your system lately, fstab, etc? |
nothing since the previous, successful sync
| Anon-E-moose wrote: | | If you do "touch /usr/portage/test.file" is there a file called "test.file" in /usr/portage? |
yes |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 2:51 pm Post subject: |
|
|
| equaeghe wrote: | | I've changed my rsync mirror to the US one from the European one and am now syncing using the gatech mirror (goes very slowly!). Will report back once it's done. |
My tree is still empty, but now the error was different:
| Code: | sent 2.71M bytes received 228.14M bytes 33.38K bytes/sec
total size is 218.82M speedup is 0.95
* Manifest timestamp: 2018-09-18 12:08:42 UTC
* Valid OpenPGP signature found:
* - primary key: DCD05B71EAB94199527F44ACDB6B8C1F96D8BF6D
* - subkey: E1D6ABB63BFCFB4BA02FDF1CEC590EEAC9189250
* - timestamp: 2018-09-18 12:08:42 UTC
* Verifying /usr/portage/.tmp-unverified-download-quarantine ...!!! Manifest verification failed:
Manifest mismatch for metadata/Manifest.gz
__size__: expected: 1982, have: 1979
q: Updating ebuild cache in /usr/portage ...
q: Finished 227 entries in 0.005788 seconds
Action: sync for repo: gentoo, returned code = 1
* An update to portage is available. It is _highly_ recommended
* that you update portage now, before any other packages are updated.
* To update portage, run 'emerge --oneshot portage' now.
* emerge --sync failed
* Time statistics:
6922 seconds for syncing
6924 seconds total
|
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Tue Sep 18, 2018 2:59 pm Post subject: |
|
|
What is your portage version "emerge --version"
and what does "ls -la /usr/portage" return
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 3:26 pm Post subject: |
|
|
| Anon-E-moose wrote: | | What is your portage version "emerge --version" |
| Code: | # emerge --version
Portage 2.3.49 (python 3.6.5-final-0, unavailable, gcc-7.3.0, glibc-2.26-r7, 4.14.65-gentoo x86_64) |
| Anon-E-moose wrote: | | what does "ls -la /usr/portage" return |
output of exa -laT -L 2 --group-directories-first /usr/portage/: https://pastebin.com/raw/wR427Li1
So the tree is not entirely empty, as some app-a… categories have some content. The rest is stuffed into .tmp-unverified-download-quarantine for some reason. |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Tue Sep 18, 2018 4:01 pm Post subject: |
|
|
I would delete everything under /usr/portage, that includes .tmp-unverified-download-quarantine and stuff under it.
Do an "ls -la /usr/portage" and make sure there are absolutely no files there including the hidden ones.
Then run emerge-webrsync and see if that doesn't give you a complete /usr/portage
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 4:26 pm Post subject: |
|
|
| Anon-E-moose wrote: | I would delete everything under /usr/portage, that includes .tmp-unverified-download-quarantine and stuff under it.
Do an "ls -la /usr/portage" and make sure there are absolutely no files there including the hidden ones.
Then run emerge-webrsync and see if that doesn't give you a complete /usr/portage |
It does. But when I then sync again, the original issue reappears. |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Tue Sep 18, 2018 5:03 pm Post subject: |
|
|
| equaeghe wrote: | | Anon-E-moose wrote: | I would delete everything under /usr/portage, that includes .tmp-unverified-download-quarantine and stuff under it.
Do an "ls -la /usr/portage" and make sure there are absolutely no files there including the hidden ones.
Then run emerge-webrsync and see if that doesn't give you a complete /usr/portage |
It does. But when I then sync again, the original issue reappears. |
Don't do a sync, doing the webrsync should do the same thing.
After you do the webrsync I would do "emerge portage" then "emerge @world" and see if things don't start working properly.
If that doesn't work then you've done something horribly wrong to your portage environment and I can't help past this point.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Tue Sep 18, 2018 8:36 pm Post subject: |
|
|
| Anon-E-moose wrote: | | Don't do a sync, doing the webrsync should do the same thing. |
I did the webrsync and then subsequently a sync, it's the second one that messes things up.
| Anon-E-moose wrote: | | After you do the webrsync I would do "emerge portage" then "emerge @world" and see if things don't start working properly. |
I've done the portage rebuild, but I'm refraining from rebuilding my whole world, as I don't see what that would change. I'm waiting a while to see if someone else knows more. The .tmp-unverified-download-quarantine thing must be related somehow.
| Anon-E-moose wrote: | | If that doesn't work then you've done something horribly wrong to your portage environment and I can't help past this point. |
I can understand. You've already assisted me very kindly, thanks! (I do want to state that I did not mess up my system: I synced yesterday without issues, did nothing as a superuser in between and synced again today with the bad result.) |
|
| Back to top |
|
|
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Tue Sep 18, 2018 10:59 pm Post subject: |
|
|
| equaeghe wrote: | | The .tmp-unverified-download-quarantine thing must be related somehow. |
equaeghe ... this is the result of 'sync-allow-hardlinks = yes' (which was introduced and is the default), to disable it set 'sync-allow-hardlinks = no' in repos.conf (see the above linked, or 'eselect news', 2018-07-11-portage-sync-allow-hardlinks).
best ... khay |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Wed Sep 19, 2018 9:45 am Post subject: |
|
|
| equaeghe wrote: | | Anon-E-moose wrote: | | If that doesn't work then you've done something horribly wrong to your portage environment and I can't help past this point. |
I can understand. You've already assisted me very kindly, thanks! (I do want to state that I did not mess up my system: I synced yesterday without issues, did nothing as a superuser in between and synced again today with the bad result.) |
I owe you an apology on that, as what happened to you happened to me last night,
and there's nothing wrong with hard links on my system, but I set the no flag in repos.conf since.
It seems the devs are bound and determined to destroy gentoo one way or another, "hey lets screw with whats been working forever" 
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
Fitzcarraldo
Advocate
Joined: 30 Aug 2008
Posts: 2034
Location: United Kingdom
|
| Posted: Wed Sep 19, 2018 11:15 am Post subject: |
|
|
I'm trying to understand this thread before I sync my two Gentoo installations this week.
Anon-E-moose and equaeghe, are you saying you do not have an overlay filesystem (yet you nevertheless have a problem syncing)?
Excerpt from https://www.gentoo.org/support/news-items/2018-07-11-portage-sync-allow-hardlinks.html :
| Zac Medico wrote: | | The new behavior may conflict with configurations that restrict the use of hardlinks, such as overlay filesystems. Therefore, users will have to set "sync-allow-hardlinks = no" in repos.conf if they have a configuration that restricts the use of hardlinks, but this should not be very common |
_________________
Clevo W230SS: amd64, VIDEO_CARDS="intel modesetting nvidia".
Compal NBLB2: ~amd64, xf86-video-ati. Dual boot Win 7 Pro 64-bit.
OpenRC udev elogind & KDE on both.
Fitzcarraldo's blog |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
| Posted: Wed Sep 19, 2018 11:53 am Post subject: |
|
|
| Fitzcarraldo wrote: | | are you saying you do not have an overlay filesystem (yet you nevertheless have a problem syncing)? |
Yes. I will file a bug and report its number here.
| khayyam wrote: | | set 'sync-allow-hardlinks = no' in repos.conf |
I tried this, and it works. Thanks! (Of course this is just a workaround; I guess I'll move to git syncing, where all the attention is nowadays.) |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6098
Location: Dallas area
|
| Posted: Wed Sep 19, 2018 11:56 am Post subject: |
|
|
Fitzcarraldo, I don't use overlay at all, it's not even set in the kernel.
| Code: | * Verifying /usr/portage/.tmp-unverified-download-quarantine ...!!! Manifest verification failed:
Manifest mismatch for metadata/Manifest.gz
__size__: expected: 1981, have: 1980 |
Day before yesterday it was fine, this morning's sync showed the problem.
so I set hardlinks no in repos.conf and resync'd, showed clean then.
Whichever dev came up with this idea and claimed that most wouldn't be affected as it's not common is a little wrong.
Edit to add: The problem is that any error will trigger off unwanted behavior, things left in .tmp-unverified-download-quarantine, main /usr/portage/* left in a weird and possibly unstable state. As is shown above it was simply a one byte off mismatch that triggered my problem.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Last edited by Anon-E-moose on Wed Sep 19, 2018 9:00 pm; edited 2 times in total |
|
| Back to top |
|
|
equaeghe
l33t
Joined: 22 Feb 2005
Posts: 637
|
|
| Back to top |
|
|
|
Portage & Programming
