| View previous topic :: View next topic |
| Author |
Message |
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
 Posted: Thu Aug 11, 2022 10:11 pm Post subject: binary client upgrade version confustion |
 |
|
I'd like to understand why portage seems confused about versions for some packages on the client while the chroot has no such problems.
I was able to add some package masks to force the issue, but that doesn't seem like a good solution unless there is a path that explains why.
What follows is a close reconstruction of events.
Output from the client fetching binaries. Exhibits the initial problem
https://dpaste.com/E62TVQWSJ
I can't decipher why it wants to downgrade packages and reinstall dev-python/appdirs with old python versions.
I migrated (seemingly) successfully to python 3.10 a while back, but I left the old python versions installed on the client for a while. I uninstalled the old versions of python, but I don't believe I tried fetching updates before uninstalling.
Installs / uninstalls can be observed in emerge.log (Aug 10 & 11):
https://dpaste.com/6V9V7CEFL
A reverse chronologically annotated package.mask file to force correct versions (begins at bottom):
https://dpaste.com/GS9VZGTYQ
What would a reinstall do for those packages for the chroot and client:
https://dpaste.com/78MJLRNXL
emerge --info for
chroot: https://dpaste.com/59EKFHNNS
client: https://dpaste.com/5EBB4K5MY
I did remove some of the binary packages from the client, the chroot, and the BINHOST (I don't recall which, but it didn't have an impact). Here is the result of find for related binary packages on the client:
https://dpaste.com/GL56UZHEY
Additionally, while adding the most recent package mask entries, Portage seemed to have an odd knowledge of historical gentoo-syntax versions, resulting in the long list of single entry masks.
EDIT:
With the package masks commented out, here is what a current fetch looks like (might be the same as the previous one):
https://dpaste.com/8T6PEQ43R
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
 |
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21591
|
| Posted: Fri Aug 12, 2022 3:10 pm Post subject: |
|
|
| Regarding the last paste: your binhost server is offering dev-python/pycparser built for pre-3.10 Python. If this were installed, it would require a setuptools built for pre-3.10 Python. That causes a conflict, since you have other things that want setuptools built for 3.10. You need the client not to install dev-python/pycparser or the binhost server to stop offering that pycparser, whether by deleting it from PKGDIR or by rebuilding it with Python 3.10. If I had to guess, the binhost has no further use for dev-python/pycparser, and may even have uninstalled it entirely. Then, when you migrated to Python 3.10, dev-python/pycparser would not have been rebuilt, so the old Python 3.8 + 3.9 version would remain in PKGDIR as a trap for client machines. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Fri Aug 12, 2022 11:12 pm Post subject: |
|
|
Hmm. I was hopeful, but I'm not sure.
pycparser is installed in the chroot, and is required by cffi, which in turn is required by 4 other packages. client matches.
chroot and client Installed versions, respectively, agree: | Code: | | [ebuild R ] dev-python/pycparser-2.21-r1::gentoo USE="-test" PYTHON_TARGETS="python3_10 (-pypy3) -python3_8 -python3_9 (-python3_11)" 0 KiB |
| Code: | | [ebuild R ] dev-python/pycparser-2.21-r1::gentoo USE="-test" PYTHON_TARGETS="python3_10 (-pypy3) -python3_8 -python3_9 (-python3_11)" 167 KiB |
The chroot, BINHOST and client have the same binary packages by version and sha512sum: | chroot: | # sha256sum dev-python/pycparser-2.2*
e7cfaf3b087bab2844ed083be0c2364cce688772106d4936b0e7863aa801bb4d dev-python/pycparser-2.20-r1.tbz2
ede26abe6517fba857898e814c74806def5dcc712479aee7b89b06d379f8253a dev-python/pycparser-2.21-r1.tbz2
0cabd3d1584470835ff4663a5466de06fb0229743c14e78e82bb4cfbca660615 dev-python/pycparser-2.21.tbz2 |
| binhost: | $ sha256sum dev-python/pycparser-2.2*
e7cfaf3b087bab2844ed083be0c2364cce688772106d4936b0e7863aa801bb4d dev-python/pycparser-2.20-r1.tbz2
ede26abe6517fba857898e814c74806def5dcc712479aee7b89b06d379f8253a dev-python/pycparser-2.21-r1.tbz2
0cabd3d1584470835ff4663a5466de06fb0229743c14e78e82bb4cfbca660615 dev-python/pycparser-2.21.tbz2 |
| client: | $ sha256sum dev-python/pycparser-2.2*
e7cfaf3b087bab2844ed083be0c2364cce688772106d4936b0e7863aa801bb4d dev-python/pycparser-2.20-r1.tbz2
ede26abe6517fba857898e814c74806def5dcc712479aee7b89b06d379f8253a dev-python/pycparser-2.21-r1.tbz2
0cabd3d1584470835ff4663a5466de06fb0229743c14e78e82bb4cfbca660615 dev-python/pycparser-2.21.tbz2 |
I can delete the old binpkgs versions from the BINHOST and all form the client, that should force it to see the "correct" version. If that doesn't work, I can try rebuilding it in the chroot and repeating, but how it would have the wrong python versions in the installed package seems unlikely?
It will be later this evening before I can do that, but I'll post back with the results. I'll also take a closer look at other packages to make sure there isn't some other package that's causing the issue.
Thanks!
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21591
|
| Posted: Fri Aug 12, 2022 11:19 pm Post subject: |
|
|
| Your installed version is 2.21-r1. The paste at the bottom of your first post says Portage would install 2.21 (no -r1). You may have the same tbz2 for 2.21 in all three places, and have it be for the old Python in all three places. Perhaps you should mask <=pycparser-2.21, to force Portage not to downgrade from your correctly built 2.21-r1. That may provoke new error messages, but hopefully those will be more informative messages. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Sat Aug 13, 2022 5:57 am Post subject: |
|
|
Some of what is interesting are the versions it thinks it wants to use that don't exist in ::gentoo.
efivar-37-r1, gentoo-syntax-20211208 (or any version not "-2"), nss-3.70 or xcffib-0.11.1. cffi has both versions (installed and downgrade).
Here is the mask: | Code: | | <=dev-python/pycparser-2.21 |
Here is the output of emerge -uDgKfp --with-bdeps=y --backtrack=42 --verbose-conflicts @world:
https://dpaste.com/HFWJJPWZS
Among those packages, here are all that remain on the client: | Code: | $ find /var/repos/binpkgs/ -type f -name "efivar-37*" -o -name "nss3.70*" -o -name "gentoo-syntax*" -o -name "cffi-1.15*" -o -name "xcffib-0.11*" -o -name "pycparser" -o -name "appdirs*"
/var/repos/binpkgs/dev-python/xcffib-0.11.1-r1.tbz2
/var/repos/binpkgs/dev-python/cffi-1.15.1.tbz2
/var/repos/binpkgs/app-vim/gentoo-syntax-2.tbz2
/var/repos/binpkgs/sys-libs/efivar-37.tbz2 |
I used eclean to identify a bunch of older packages, but that didn't seem to help. I did it manually, so not everything eclean suggested was removed. I haven't yet done that on the BINHOST.
Here is what portage has: | Code: | $ find /var/repos/gentoo{,/metadata/} -maxdepth 3 -name "efivar-37*" -o -name "nss-3.*" -o -name "gentoo-syntax-*" -o -name "cffi-1.15*" -o -name "xcffib-0.11*" -o -name "pycparser-*" -o -name "appdirs-*"
/var/repos/gentoo/app-vim/gentoo-syntax/gentoo-syntax-2.ebuild
/var/repos/gentoo/app-vim/gentoo-syntax/gentoo-syntax-9999.ebuild
/var/repos/gentoo/dev-libs/nss/nss-3.68.4.ebuild
/var/repos/gentoo/dev-libs/nss/nss-3.79-r1.ebuild
/var/repos/gentoo/dev-libs/nss/nss-3.81.ebuild
/var/repos/gentoo/dev-python/appdirs/appdirs-1.4.4-r2.ebuild
/var/repos/gentoo/dev-python/cffi/cffi-1.15.0-r1.ebuild
/var/repos/gentoo/dev-python/cffi/cffi-1.15.1.ebuild
/var/repos/gentoo/dev-python/pycparser/pycparser-2.21-r1.ebuild
/var/repos/gentoo/dev-python/xcffib/xcffib-0.11.1-r1.ebuild
/var/repos/gentoo/sys-libs/efivar/efivar-37.ebuild
/var/repos/gentoo/metadata/md5-cache/app-vim/gentoo-syntax-2
/var/repos/gentoo/metadata/md5-cache/app-vim/gentoo-syntax-9999
/var/repos/gentoo/metadata/md5-cache/dev-libs/nss-3.68.4
/var/repos/gentoo/metadata/md5-cache/dev-libs/nss-3.79-r1
/var/repos/gentoo/metadata/md5-cache/dev-libs/nss-3.81
/var/repos/gentoo/metadata/md5-cache/dev-python/appdirs-1.4.4-r2
/var/repos/gentoo/metadata/md5-cache/dev-python/cffi-1.15.0-r1
/var/repos/gentoo/metadata/md5-cache/dev-python/cffi-1.15.1
/var/repos/gentoo/metadata/md5-cache/dev-python/pycparser-2.21-r1
/var/repos/gentoo/metadata/md5-cache/dev-python/xcffib-0.11.1-r1
/var/repos/gentoo/metadata/md5-cache/sys-libs/efivar-37 |
It looks to me like Portage is having trouble figuring out what to do without downgrading. And that doesn't make a lot of sense if not downgrading works (based on adding a mask for all of those versions). Which seems "broken," since the configuration between the chroot and client would be different.
I guess I can try digging through the contents of binary packages to see if they contain reference to which version of python they were built against, but that doesn't seem like a maintainable solution.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21591
|
| Posted: Sat Aug 13, 2022 3:30 pm Post subject: |
|
|
| pjp wrote: | | Some of what is interesting are the versions it thinks it wants to use that don't exist in ::gentoo. |
When installing a binary package, the corresponding ebuild is not required to exist. This can enable you to install packages that have expired from the tree, or that exist in a newer tree that the binhost can see and the client cannot see yet. | pjp wrote: | Here is the mask: | Code: | | <=dev-python/pycparser-2.21 |
|
That looks right. | pjp wrote: | | Among those packages, here are all that remain on the client: |
Depending on how you invoke emerge, it may be able to fetch packages from the server at need, in which case the list you show here is not necessarily complete. | pjp wrote: | | I did it manually, so not everything eclean suggested was removed. I haven't yet done that on the BINHOST. |
I think you need to clean both if you want to fully eliminate the client's ability to consider old versions. | pjp wrote: | | It looks to me like Portage is having trouble figuring out what to do without downgrading. |
I agree, the shown output is strange. | pjp wrote: | | And that doesn't make a lot of sense if not downgrading works (based on adding a mask for all of those versions). Which seems "broken," since the configuration between the chroot and client would be different. |
Perhaps the client has packages that the chroot lacks, and one of those is encouraging the bad configuration. | pjp wrote: | | I guess I can try digging through the contents of binary packages to see if they contain reference to which version of python they were built against, but that doesn't seem like a maintainable solution. |
They will contain it.
To be sure I understand: if you restore the masks of old versions, the client successfully picks the new versions, and even a full emerge --update @world is clean. If you then revoke those masks, the client starts trying to downgrade again. Is that description accurate? If so, I agree that something strange is happening. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Sun Aug 14, 2022 6:01 am Post subject: |
|
|
| Hu wrote: | | When installing a binary package, the corresponding ebuild is not required to exist. This can enable you to install packages that have expired from the tree, or that exist in a newer tree that the binhost can see and the client cannot see yet. |
That makes sense. It's still surprising to see with no visible means of how it is requiring those versions.
| Hu wrote: | | Depending on how you invoke emerge, it may be able to fetch packages from the server at need, in which case the list you show here is not necessarily complete. |
I rely primarily on options "g" (get binaries from BINHOST) and "K" (only use packages from PKGDIR). More specifically, "uDgKf" followed by "uDgK". That has seemed to work by first fetching, then installing locally. Excluding some problems early on in my use of a BINHOST, those options have worked without issues until now.
| Hu wrote: | | I think you need to clean both if you want to fully eliminate the client's ability to consider old versions. |
Yeah, I considered that might be necessary. Given my description above as to how it has worked until now, I was hoping that wasn't going to be necessary. As far as I can see, there's no reason reported by Portage as to why it is choosing the versions. python 3.10 is exclusively used, seemingly with nothing clearly having a dependency of 3.8 and / or 3.9.
| Hu wrote: | | I agree, the shown output is strange. |
That's at least reassuring.
| Hu wrote: | | Perhaps the client has packages that the chroot lacks, and one of those is encouraging the bad configuration. |
I've wondered that as well, but not until I read your comment did I realize I should make a comparison. I'll do that next.
| Hu wrote: | | They will contain it. |
I thought so, but wasn't sure. I was hoping to not have to do that, but it's looking more and more likely.
| Hu wrote: | | To be sure I understand: if you restore the masks of old versions, the client successfully picks the new versions, and even a full emerge --update @world is clean. If you then revoke those masks, the client starts trying to downgrade again. Is that description accurate? If so, I agree that something strange is happening. |
To clarify, masks are required for some old and some newer versions. 2 examples for newer versions are efivar where it wants a -r version, and gentoo-syntax, which is merely a confusion about the current version "2" being lower than year based versions which no longer appear to be used.
Otherwise, yes, the output is clean. Using the following refined mask, "emerge -uDgKtp --with-bdeps=y --backtrack=42 --verbose-conflicts --complete-graph @world" produces "Calculating dependencies... done!" | Code: | <=dev-python/pycparser-2.21
<=dev-python/xcffib-0.11.1
=dev-libs/nss-3.70
=sys-libs/efivar-37-r1
>=app-vim/gentoo-syntax-2000 |
Thanks for the feedback. I'll try comparing installed packages, then moving the client's binpkgs to see if there are any different results. After that, I may try to find what old packages are tied to python 3.8 and 3.9. That seems easier than trying to clean up old packages on the binhost. That will probably be the last resort.
EDIT:
Virtually identical packages. openrc versions differ. Client has a newer version, which also includes a virtual/pam.
Moving client's binpkgs directory had no effect without using masks (produces the same issue expecting python 3.8 / 3.9).
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
