PYTHON TARGETS switch, anyone else nervous today?

[krumpf]

There's only 1 package that's preventing me from removing python:3.8, app-misc/piper hasn't been updated to python:3.9. Guess I'll have to wait until the ebuild gets updated

[Ionen]

Code: Select all

Calculating dependencies... done!
  dev-lang/python-3.8.9_p2 pulled in by:
    x11-misc/menumaker-0.99.12 requires dev-lang/python:3.8

I used menumaker long ago to help set up openbox. After removing menumaker, 'emerge --depclean' removed python-3.8.
(It left one preserved libs that 'emerge @preserved-rebuild' took care of.)

Given it's maintainer-needed, added 3.9 myself in case ever want it back

There's only 1 package that's preventing me from removing python:3.8, app-misc/piper hasn't been updated to python:3.9. Guess I'll have to wait until the ebuild gets updated

Dropped a side-note about that one in bug #792732 few days ago, hopefully gets attention.

[Errelin]

I tried to replace my Python target too. Generally just about 60 pkgs to go and all succeeded except spidermonkey.

Spidermonkey, along with rust, clang and firefox, has been giving me hard times on a ~amd64 testing branch. So I masked them all for now and will upgrade them manually when I see a stable version is released.

Now I've been much used to such compile failure. I'm currently with Spidermonkey 78.10.1: 78, and the lastest is 78.11.0 : 78 which seems to be under testing: https://packages.gentoo.org/packages/de ... idermonkey

In terms of removal of Python:3.8, it seems quite some pkgs on my machine need it: readline, openssl, sqlite, zlib and so on. But if i run, say, qdepends sys-libs/readline, it depends not on Python 3.8 instead ... I'm confused.

Should I unmerge them and rebuild them using Python 3.9 or just wait for Portage to care when the ebuilds get updated?

[stonespheres]

I tried to replace my Python target too. Generally just about 60 pkgs to go and all succeeded except spidermonkey.

Spidermonkey, along with rust, clang and firefox, has been giving me hard times on a ~amd64 testing branch. So I masked them all for now and will upgrade them manually when I see a stable version is released.

Now I've been much used to such compile failure. I'm currently with Spidermonkey 78.10.1: 78, and the lastest is 78.11.0 : 78 which seems to be under testing: https://packages.gentoo.org/packages/de ... idermonkey

In terms of removal of Python:3.8, it seems quite some pkgs on my machine need it: readline, openssl, sqlite, zlib and so on. But if i run, say, qdepends sys-libs/readline, it depends not on Python 3.8 instead ... I'm confused.

Should I unmerge them and rebuild them using Python 3.9 or just wait for Portage to care when the ebuilds get updated?

Hmm, all of those packages are specifically large. Do you have them set to compile in notmpfs/do you have enough RAM to compile those?

[Errelin]

Hmm, all of those packages are specifically large. Do you have them set to compile in notmpfs/do you have enough RAM to compile those?

Hi, I think RAM is not a big problem since I got 64GB and was running update process in tty. I failed to compile clang, llvm, rust in the past.

I've never set PYTHON_TARGETS stuff in my make.conf so this time it seems both 3.9 and 3.10beta have been pulled in. The very question I care most now is:
How should I remove python 3.8 since depclean reports other pkgs except Python 3.8 ....

I read up the thread but found it was a bit overwhelming ... Thanks for your reply.

[Ionen]

Hmm, all of those packages are specifically large. Do you have them set to compile in notmpfs/do you have enough RAM to compile those?

Hi, I think RAM is not a big problem since I got 64GB and was running update process in tty.

Well, depending on settings I wouldn't entirely rule it out. rust for one can use 8-30+GB of space on a tmpfs depending on USE, each rustc process can use something between ~1-4.5GB (perhaps more lately), and multiply that by -jX value -- can very well go over 64GB if it's high enough.

That aside, dmesg output would have mention something near the end if it's what happened in most cases.

Hard to say what else might've happened without seeing some build logs though. Hoping it's not internal compiler errors / segfaults.

[Errelin]

Well, depending on settings I wouldn't entirely rule it out. rust for one can use 8-30+GB of space on a tmpfs depending on USE, each rustc process can use something between ~1-4.5GB (perhaps more lately), and multiply that by -jX value -- can very well go over 64GB if it's high enough.

That aside, dmesg output would have mention something near the end if it's what happened in most cases.

Hard to say what else might've happened without seeing some build logs though. Hoping it's not internal compiler errors / segfaults.

I totally agree on this and this is the reason I set only -j8 for MAKEOPTS with 64GB ram. Nothing more. 64GB is the max my motherboard can support, sigh. I definitely desire more.

I declined to post the build.log and other USE info cuz I think such failure should be expected considering I'm on a testing branch system-wide. This time, the error looks very like the one mentioned in this thread earlier by the guy who failed at qtwebengine: emake failed or sth like that.

And since this thread is more about Python upgrade, I'll try to focus on upgrading @world with Python 3.9 (already done), and remove 3.8 (still trying to figure this out on my machine). Thanks for your reply.

[cameta]

cameta,
maybe ran you in out of memory error? you can check with dmesg. If isn't the case post your emerge --info and the build.log.

The rebuild of many package, including qtwebengine, is due to the stabilization of icu-69 required by new libreoffice-bin version.

Yes, this was the problem. Nothing to do with the Python update.
viewtopic-t-1131918-highlight-.html

[cameta]

I tried to replace my Python target too. Generally just about 60 pkgs to go and all succeeded except spidermonkey.

Spidermonkey, along with rust, clang and firefox, has been giving me hard times on a ~amd64 testing branch. So I masked them all for now and will upgrade them manually when I see a stable version is released.

Now I've been much used to such compile failure. I'm currently with Spidermonkey 78.10.1: 78, and the lastest is 78.11.0 : 78 which seems to be under testing: https://packages.gentoo.org/packages/de ... idermonkey

In terms of removal of Python:3.8, it seems quite some pkgs on my machine need it: readline, openssl, sqlite, zlib and so on. But if i run, say, qdepends sys-libs/readline, it depends not on Python 3.8 instead ... I'm confused.

Should I unmerge them and rebuild them using Python 3.9 or just wait for Portage to care when the ebuilds get updated?

Look at this problem
viewtopic-t-1136421.html

[figueroa]

Up-to-date two days ago remote x86 server updated today. Nothing python related in /etc/portage/make.conf or /etc/portage/package.use. It updated 82 packages, mostly for Python 3.9, but also gcc-10.3.0, and then Python 3.8 and one other package deplceaned away cleanly. No fuss.

[Errelin]

Look at this problem
viewtopic-t-1136421.html

Hi cameta, thanks for your information. I failed to locate /usr/lib64/rustlib on my machine however. I still believe my problem was caused by an unstable spidermonkey (the latest, under early testing). So I'm fine with the one already merged. I'll keep this in mind in case I need to look into this lib dir in the future.


Also I figured out why Python:3.8 gets stuck in my Gentoo:
1. run

Code: Select all

emerge -pcv python:3.8

I got a short list of ebuilds that depend on python:3.8. Among which there are portage, libxml2. At least for portage I can recall that I once updated it with --oneshot switch.

2. run

Code: Select all

emerge -pv portage

or any ebuild listed as a result of the above step, I saw PYTHON_TARGETS="python3_8 python3_9"!

Since I'm using a directory package.use, rather than a single file, and the global PYTHON_TARGETS stuff was not in my make.conf, but in /etc/portage/package.use/world. Thus, the updated targets will not be applied to any ebuild that is not included in @world, and consequently updating @world will not affect those outside of the set ...

Hope whoever has the same confusion can check in the way suggested and remove python3_8 as needed.

[Hu]

Since I'm using a directory package.use, rather than a single file, and the global PYTHON_TARGETS stuff was not in my make.conf, but in /etc/portage/package.use/world. Thus, the updated targets will not be applied to any ebuild that is not included in @world, and consequently updating @world will not affect those outside of the set ...

Assuming that your setting in /etc/portage/package.use/world is:

Code: Select all

*/* PYTHON_TARGETS: python_targets_my_favorite_version

Then it will affect all packages that are rebuilt, regardless of whether they are in @world. However, as you hinted at, if a package is not in @world, then depending on the Portage options used, the package may not be scheduled for a rebuild. If it does not rebuild, it cannot have its settings changed.

[pietinger]

[...] Nothing python related in /etc/portage/make.conf or /etc/portage/package.use. It updated [X] packages, mostly for Python 3.9, but also gcc-10.3.0, and then Python 3.8 and one other package deplceaned away cleanly. No fuss.

+ 1

X=135 (+ gentoo-sources-5.10.42 + qtwebengine + libreoffice ... )

All perfect !

[Errelin]

Assuming that your setting in /etc/portage/package.use/world is:

Code: Select all

*/* PYTHON_TARGETS: python_targets_my_favorite_version

Then it will affect all packages that are rebuilt, regardless of whether they are in @world. However, as you hinted at, if a package is not in @world, then depending on the Portage options used, the package may not be scheduled for a rebuild. If it does not rebuild, it cannot have its settings changed.

Yes, creating a file called "world" or "base" is something I learned while searching through this forum for topics/discussions on package.use: a file or a dir? And I have just done that. The last thing I found about python:3.8 was a few dependencies pulled in by vim still had python_targets_python3_8 flag, so I removed all such flags and then --depclean again. This time only python:3.8 was to be cleaned. Thanks for pointing out that in this thread. It is quite a useful technique for those who like to use a dir as the USE control switch.

[eccerr0r]

Look like that black sheep for me is fio, else 3.8 would no longer be a dep on my machine.

[figueroa]

This fio?

Code: Select all

$ emerge -p sys-block/fio

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] sys-block/fio-3.20-r2  USE="gtk zlib -aio -curl -glusterfs -gnuplot -io-uring -numa -python -rbd -rdma (-static) -tcmalloc -test -zbc" PYTHON_TARGETS="-python3_8" 

[Anon-E-moose]

Code: Select all

Calculating dependencies... done!
[ebuild     U  ] sys-block/fio-3.27::gentoo [3.20-r2::gentoo] USE="zlib -aio -curl -glusterfs -gnuplot -gtk -io-uring -numa -python -rbd -rdma (-static) -tcmalloc -test -zbc" PYTHON_TARGETS="python3_9%* -python3_8 (-python3_7%)" 938 KiB

[eccerr0r]

Oh wait nevermind, yeah fio-3.20-r2 requires python 3.7 or 3.8, fio-3.27 is ~amd64 on my machine and can be used with python3.9. hmm... Either way it shouldn't be too bad to workaround...