Stuck in python 3.12 upgrade I believe.

[tckosvic]

Just an engineer; not IT guy. Short-time gentoo user. Routine upgrades done weeky with emerge line as below:

Code: Select all

emerge --update --deep --changed-use --exclude gentoo-kernel-bin --with-bdeps=y @world  -av

Today (6/8) after python update attempt, I am having USE flag messages from emerge that I don't understand that don't allow emerge to run. See below:

Code: Select all

gentooVM ~ # emerge --update --deep --changed-use --exclude gentoo-kernel-bin --with-bdeps=y @world  -av 

These are the packages that would be merged, in order:

Calculating dependencies                                 n

!!! Problem resolving dependencies for app-editors/gedit from @selected                                                                                            ... done!
Dependency resolution took 2.14 s (backtrack: 0/20).


!!! The ebuild selected to satisfy "app-editors/gedit" has unmet requirements.
- app-editors/gedit-46.2::gentoo USE="python -gtk-doc" ABI_X86="(64)" PYTHON_SINGLE_TARGET="python3_11 python3_12 -python3_10"

  The following REQUIRED_USE flag constraints are unsatisfied:
    python? ( exactly-one-of ( python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 ) )

(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
gentooVM ~ #

I have added each of: python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 separately as USE into /etc/portage/make.conf and still repeatedly get same output.

My USE flage set is as below:

Code: Select all

USE="X gtk qt5 proprietary-codecs widevine nls 64 lua5-4 mmx sse alsa \
     pulseaudio bluetooth jack json vtk widgets webengine webchannel \
     concurrent gui network printsupport sql svg testlib xml gles2 qml \
     positioning chromium opus quick webp zstd lzma hdf5 pcre32 minimal caps \
     custom-modes cxx geos abi_x86_64 szip python_single_target_python3_11 \
     script -minimal tools pkcs11 fortran gtk3 apache2_modules_auth_digest jumbo-build "

Need suggestions to get emerge going. As I said, I tried each python version as a USE flag If more info is needed let me know.

I read the news item on the python change and couldn't understand where to put the code snippets related to python listed in there.

thanks, tom kosvic

[freke]

python_single_target nor python_targets belongs in make.conf - at best in package.use.

Only one python_single_target is allowed for an ebuild - it complains about multiple single targets set.

[tckosvic]

@freke, I do not understand your comments. I added "python_single_target_python3_11" to USE as I saw it in emerge output. Was not there before I got these messages.
I guess I am confused about where do I have multiple python targets set.

[logrusx]

@freke, I do not understand your comments. I added "python_single_target_python3_11" to USE as I saw it in emerge output. Was not there before I got these messages.
I guess I am confused about where do I have multiple python targets set.

You should not arbitrarily add use flags in make.conf. Rather do it per package in /etc/portage/package.use Read the news item for python 12 upgrade instructions.

Python single target should contain only a single python target, hence the name - PYTHON_SINGLE_TARGET. That's what portage complains for. What you've done putting it in make.conf is make PYTHON_SINGLE_TARGET contain two targets.

Once again, make.conf is not the place to put use flags, especially to fix stuff.

Best Regards,
Georgi

[pietinger]

The news from 2024-05-09-python3-12 says:

If you have PYTHON_TARGETS or PYTHON_SINGLE_TARGET declared
in make.conf, please remove these declarations as they will interfere
with the package.use samples provided below. Using make.conf for Python
targets is discouraged as it prevents package defaults from applying
when necessary. This news item assumes using /etc/portage/package.use
or your package manager's equivalent file for configuration.

In the old days /etc/portage/package.use was a file - today it is a directory. Create in this directory a file (the file name does not matter) and be sure to not forget the red marked ones (without portage would add existing definitions together):

*/* PYTHON_TARGETS: -* python3_11 python3_12
*/* PYTHON_SINGLE_TARGET: -* python3_12

Then try again your world-update.

[figueroa]

pietinger
Mine is a file, by choice, even in my most recent install. I find it much easier to edit a single file with the entries in alphabetic order, especially since sometimes entries are closely related, as when trying to migrate from one python version to another.

[pietinger]

pietinger
Mine is a file, by choice, even in my most recent install. I find it much easier to edit a single file with the entries in alphabetic order, especially since sometimes entries are closely related, as when trying to migrate from one python version to another.

figueroa,
mine is also a file (even in my most recent install) I love it to see everything at once

[logrusx]

pietinger
Mine is a file, by choice, even in my most recent install. I find it much easier to edit a single file with the entries in alphabetic order, especially since sometimes entries are closely related, as when trying to migrate from one python version to another.

figueroa,
mine is also a file (even in my most recent install) :lol: I love it to see everything at once 8)

You can still have a single file in the directory. However this gives you the opportunity to have other files, say you're trying something and you give the file a specific name to remind you of its purpose. After you're done with the trial, you can get rid of the file altogether, rather than having to edit your main file.

You can even have other directories with other files inside them. I've tested it that way, but settled for files.

In the same time if you're doing something complicated with use flags (be it organization or process), that's a design smell which should make whoever is doing that to take a step back and reconsider their decision.

Best Regards,
Georgi

[Josef.95]

tckosvic,

in doubt, best is set nothing.
With this, portage will using the sane defaults from profile - this should work.

[tckosvic]

This is totally confusing issue and I have made no progress. I'll respond to some of the comments I have received.

I put python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 each separately into make.conf USE because emerge very specifically said to. Before that there was no python USE flag at all.

How could anyone know that that any one of the above USE strings was specifying python twice as was stated in the replys.

I did the steps of @pietinger. I created a new file named python.use in /etc/portage/packages.use directory. I entered the python statements supplied. I redid sync. Upon running my emerge@world string, I got the same emerge messages I presented initially. Telling me to install python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 as USE flags.

Along the way I did find out about "ufed" and used that to do some USE flag editing. It did put everything in make.conf in alphabetical order.

I do not think the wording of news item 32 re: python upgrade is very descriptive or useful.

thanks all for help, I will certainly take any additional suggestions.

[logrusx]

I put python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 each separately into make.conf USE because emerge very specifically said to.

Could you please paste output where emerge says so?

I did the steps of @pietinger. I created a new file named python.use in /etc/portage/packages.use directory. I entered the python statements supplied. I redid sync. Upon running my emerge@world string, I got the same emerge messages I presented initially. Telling me to install python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 as USE flags.

That message does not say so. It reported problems for specific package with specific use flags. If this is what you mean portage specifically told you to put python stuff into make.conf, I don't see how you made it to mean that. Are you under the impression that all use conflicts reported by portage should be solved by setting use flags in make.conf?

Along the way I did find out about "ufed" and used that to do some USE flag editing. It did put everything in make.conf in alphabetical order.

I don't know what that is, maybe it's an ancient tool that must be removed from tree. The times when the only place to set use flags was in make.conf are long gone.

I do not think the wording of news item 32 re: python upgrade is very descriptive or useful.

The wording is as useful and descriptive as it gets. You can't expect a book on migrating it with every single detail.

Best Regards,
Georgi

[tckosvic]

I have my emerge@world of 179 items running.

I modified the contents of the new python.use from:

Code: Select all

*/* PYTHON_TARGETS: -* python3_11 python3_12
*/* PYTHON_SINGLE_TARGET: -* python3_12

to

Code: Select all

*/* PYTHON_TARGETS: -* python3_11 python3_12
*/* PYTHON_SINGLE_TARGET: -* python3_11

I got this from the other running thread on python 3.12 upgrade.

A subsequent emerge run then said to add "python" as a USE (called it a tool) flag. I put "python" into make.conf . Was never there before.

Success, thus far. 179 items building

Code: Select all

  The following REQUIRED_USE flag constraints are unsatisfied:
    python? ( exactly-one-of ( python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 ) )

The above is from my initial post of my problem, to me this says REQUIRED USE.

I'll see how far this gets.

[logrusx]

A subsequent emerge run then said to add "python" as a USE (called it a tool) flag. I put "python" into make.conf . Was never there before.

Success, thus far. 179 items building

Code: Select all

  The following REQUIRED_USE flag constraints are unsatisfied:
    python? ( exactly-one-of ( python_single_target_python3_10 python_single_target_python3_11 python_single_target_python3_12 ) )

The above is from my initial post of my problem, to me this says REQUIRED USE.

I'll see how far this gets.

This is REQUIRED_USE. With capital letters and underscore. That does not mean it's asking you to put use flags in make.conf. That's a variable which should be satisfied.

PYTHON_TARGETS and PYTHON_SINGLE_TARGET are USE expand variables. That means there's special syntax to declare them as the syntax below and portage then expands their contents to individual use flags as the ones in REQUIRED_USE. REQUIRED_USE is an ebuild variable. There are certain rules that it should conform to, they are defined in the ebuild, the profile, the eclasses.

Portage will read the special syntax below for PYTHON_TARGETS as: -* - clear the the contents of the variable; python3_11 - add python_targets_python3_11 use flag; same for python3_12;
The */* is for category/package meaning all packages from all categories. That means portage will add those flags to all packages being pulled in.

The initial problem you has was you had added python_single_target_python3_11 in make.conf which combined with the one from the profile resulted in two single targets. The target should be single as the name suggests. That's why portage refused to work on that, because it doesn't know how on earth there are two single targets.

I modified the contents of the new python.use from:

Code: Select all

*/* PYTHON_TARGETS: -* python3_11 python3_12
*/* PYTHON_SINGLE_TARGET: -* python3_12

to

Code: Select all

*/* PYTHON_TARGETS: -* python3_11 python3_12
*/* PYTHON_SINGLE_TARGET: -* python3_11

Well that's good, because you corrected something that wasn't wrong but wasn't likely to work. Most often you need the single target to be of lower version as more packages support it. It should be true that all packages should support the previous default target.

Once again you should not add use flags in make.conf. There may be a few, but they are really system wide, like X or wayland. I currently have only four, all of them are with - in front, meaning I have hard unset them globally.

You should use /etc/portage/package.use

p.s.

ufed is an ncurses-based USE flag editor designed to simplify configuration of the USE flags set in make.conf. Note that ufed does not allow setting flags per-package, which is done with /etc/portage/package.use.

You should not use arbitrary tools without knowing what they do.

Best Regards,
Georgi

[tckosvic]

As I have stated several times before, python_single_target_python3_10 or python_single_target_python3_11 or python_single_target_python3_12 were NOT in make.conf when I received the emerge errors posted initially regarding python problems. In fact "python" by itself was not in make.conf. I added the above as I thought emerge was telling me to do by clearly saying "REQUIRED USE".

So I still don't know what was the initial problem. The fix was to add suggested code strings into package.use and add python as a USE flag in make.conf.

How is one to know that python is in the profile so as to cause a conflict with a python USE flag reference in make.conf. The error messages give no clue to that happening.

The news item 32 does not clearly state where to put the code snippets it is presenting. It is not clear whether they go into a file or a comandline in a terminal. Also, after 50+ years in unix and linux, I am unaware of what */* means. Must be a gentoo thing.

I do appreciate your help here, don't get me wrong.

I would like to straighten out my USE flags in make.conf. That's why I stumbled on ufed. I did read up on ufed.

Any ref you know of on how to move a large bunch of USE flags easily from make.conf into package.use files? Don't relish going through this package by package to see what USE flags are needed by each.

[Hu]

I accept that you started with none of those listed in make.conf. However, through some as yet unidentified path, you had two "single" targets concurrently set on gedit. The most likely explanation is that python3.12 came from your Gentoo Portage profile (since that one recently switched to preferring python3.12) and that you had a personal override somewhere, such as package.use, that continued to add python3.11. Adding the same target twice is not an error, so you may have had a personal setting of python3.11 for months which, because it coincided with the Gentoo Portage setting of python3.11, produced 2 uses of python3.11, and none of anything else. That would be legal. When Gentoo switched from setting python3.11 to setting python3.12, your personal addition of python3.11 became an error.

To explain the initial problem, we would need to see the state of your Portage configuration when the initial problem happened. You have made changes since that time, so it is difficult to determine what caused the initial problem.

*/* is a Gentoo thing, specifically for use in the package.* files. It applies the setting to all packages.

USE flags that do not apply to a package are ignored. Cleaning them out may make it easier to understand what your configuration does, but leaving it dirty does not hurt Portage.

[tckosvic]

I can accept that somehow I had two references to python. That is the only way to get these error messages It is lost to history now.

A few weeks ago I did the profile upgrade from 17.3 to 23. But I did several successful updates since then.

I hear you saying to leave make.conf alone as dirty and not tinker with it. Right?

One question is will I have to tinker with the strings in my new /etc/portage/package.use/python.use at some time in the future and, if so, what will I have to do?

Emerge still running fine. Will likely take a day or so even with giving it 5 cpus.

thanks again. Guess I'm whole at least for now.

[logrusx]

One question is will I have to tinker with the strings in my new /etc/portage/package.use/python.use at some time in the future and, if so, what will I have to do?

Well, you shot the fly with an RPG with those lines. Ideally it should have been only for the problematic packages. I guess emerge's message wasn't clear about what's wrong as maybe this situation may not have been expected, but in the ideal case it should have output several packages requiring use changes which you would put in a file in /etc/portage/package.use which at some point you can move somewhere else and see if an update would go without it, put it back if the update doesn't succeed. Now you can do the same thing after say a month, but when all packages are migrated emerge will have to recompile more, if not all python dependent packages.

If you went for both python and python single target for 3.12 in the beginning, the message has been correct, because portage didn't find 3.11 for some packages, it was disabled, and the mistake happen later, who knows.

You can even try to move it after the update finishes just to see what's the output. For best advise, post it here.

Best Regards,
Georgi

[Hu]

Part of the fun of Gentoo is tinkering with things. However, if Portage is happy now and you would rather not spend more time on this, leaving the files as-is should be fine for now.

Your settings in package.use/python.use will become a problem when Gentoo moves to Python 3.13 or when Gentoo removes support for an older Python that your file is keeping in use. When that happens, you will need to remove the conflicting settings. You will definitely get error messages from Portage when this becomes a problem. It will not be a silent misbehavior. My advice is to remove the PYTHON_SINGLE_TARGET override, and test to see what breaks. If you are using any packages that need Python 3.11, you may get an error from the Gentoo default of 3.12. If that happens, either (a) restore the override and wait a few weeks for Gentoo to finish the conversion to 3.12 or (b) add package-specific overrides now, and periodically revisit whether they are still needed. Longer term, you will also want to remove the PYTHON_TARGETS override. That is less critical since its presence only means your system supports 2 Python versions even when not needed. When Gentoo fully removes Python 3.11, the PYTHON_TARGETS override should become a harmless no-operation. (When Gentoo switches on Python 3.13, the PYTHON_TARGETS override will become a problem. However, I expect the 3.13 conversion is months in the future, at least.)

[tckosvic]

May as well ask my questions since I have a knowledgeable audience.

Code: Select all

My advice is to remove the PYTHON_SINGLE_TARGET override

Don't know what "single target" refers to and also what an override refers to. Does this mean to test a delete of: package.use/python.use Try an upgrade and see what happens?

Code: Select all

add package-specific overrides now

How is that done? Look at the USE flags emerge points to for each package and make a package.use file for each? How do I get past one package or do I just do many emerge runs to get through the list one package at a time?

thanks

[figueroa]

Here is a scenario for you. My /etc/portage/package.use (just a single file containing everything needed) contains a line with:

Code: Select all

media-sound/soundconverter python_single_target_python3_11

If I comment out that line, and run emerge for it, I get more output than I hoped for, but it tells me what is needed:

Code: Select all

emerge -pv media-sound/soundconverter

These are the packages that would be merged, in order:

Calculating dependencies -

!!! Problem resolving dependencies for media-sound/soundconverter
... done!
Dependency resolution took 1.83 s (backtrack: 0/20).


!!! The ebuild selected to satisfy "media-sound/soundconverter" has unmet requirements.
- media-sound/soundconverter-4.0.3::gentoo USE="aac flac mp3 ogg vorbis -opus -test" ABI_X86="(64)" PYTHON_SINGLE_TARGET="-python3_10 -python3_11"

  The following REQUIRED_USE flag constraints are unsatisfied:
    exactly-one-of ( python_single_target_python3_10 python_single_target_python3_11 )

So that tells me that the package soundconverter needs either "python_single_target_python3_11" or "python_single_target_python3_10" in /etc/portage/package.use, and you DON'T want "python_single_target_python3_10"

Currently, my /etc/portage/package.use contains the following related to the current python migration:

Code: Select all

dev-python/more-itertools python_targets_python3_11
dev-python/ordered-set python_targets_python3_11
dev-python/packaging python_targets_python3_11
dev-python/gpep517 python_targets_python3_11
dev-python/jaraco-text python_targets_python3_11
dev-python/jaraco-functools python_targets_python3_11
dev-python/jaraco-context  python_targets_python3_11
dev-python/setuptools python_targets_python3_11
dev-python/setuptools-scm python_targets_python3_11
dev-python/wheel python_targets_python3_11
dev-python/platformdirs python_targets_python3_11
app-portage/gemato  python_targets_python3_11
app-text/pastebinit python_single_target_python3_11
net-vpn/protonvpn-cli python_targets_python3_11
media-sound/soundconverter python_single_target_python3_11
dev-python/pillow python_targets_python3_11
dev-python/pygobject python_targets_python3_11
dev-python/six python_targets_python3_11

How do I know? I did trial and error by reading the output of of emerge and added them one at a time. Over time, that list will become effectively emptly. My practice is to not delete lines will in the midst of such a project as this, but to just comment them out so I can keep track of where I've been.

[tckosvic]

@fugueroa,

Need a clarification.

Code: Select all

My /etc/portage/package.use (just a single file containing everything needed) contains a line with:
Code:
media-sound/soundconverter python_single_target_python3_11

Also, it was said that:

Code: Select all

Currently, my /etc/portage/package.use contains the following related to the current python migration:
Code:
dev-python/more-itertools python_targets_python3_11
dev-python/ordered-set python_targets_python3_11
dev-python/packaging python_targets_python3_11
dev-python/gpep517 python_targets_python3_11
dev-python/jaraco-text python_targets_python3_11
dev-python/jaraco-functools python_targets_python3_11
dev-python/jaraco-context  python_targets_python3_11
dev-python/setuptools python_targets_python3_11
dev-python/setuptools-scm python_targets_python3_11
dev-python/wheel python_targets_python3_11
dev-python/platformdirs python_targets_python3_11
app-portage/gemato  python_targets_python3_11
app-text/pastebinit python_single_target_python3_11
net-vpn/protonvpn-cli python_targets_python3_11
media-sound/soundconverter python_single_target_python3_11
dev-python/pillow python_targets_python3_11
dev-python/pygobject python_targets_python3_11
dev-python/six python_targets_python3_11

Could you clarify for me?

thanks, tom kosvic

[tckosvic]

@figueroa, disregard last post. I follow what you are saying.

thanks tom kosvic

[logrusx]

Does this mean to test a delete of: package.use/python.use Try an upgrade and see what happens?

Do not delete it. Move it away, so that you can move it back. That's one convenient thing about having package.use as a directory - you can move things around.

Don't know what "single target" refers to and also what an override refers to.

This is what I found as an explanation:

A single-impl package is a package requiring the user to choose exactly one Python implementation to be built against. This means that the scripts installed by that package will be run via specified Python interpreter, and that the modules and extensions will be importable from it only. The package’s Python reverse dependencies will also have to use the same implementation. Since the package can’t support having more than one implementation enabled, its reverse dependencies have to be simple-impl as well.

Some clarification to what figueroa said:

Code: Select all

media-sound/soundconverter python_single_target_python3_11

is the expanded version of:

Code: Select all

media-sound/soundconverter PYTHON_SINGLE_TARGET: -* python3_11

Similarly for PYTHON_TARGETS. The difference being that you need -* in PYTHON_SINGLE_TARGET to clear its contents before setting a new one as it must only contain one target, while if you don't add -* it'll just aggregate what you've added in package.use.

For example the profile defines python3_12 and for a hypothetical package that supports multiple python targets you add the line:

Code: Select all

package/category PYTHON_TARGET: python3_11

it's equivalent to:

Code: Select all

package/category python_tagerts_python3_11

It'll result in final use for the package containing python_tagerts_python3_11 and python_tagerts_python3_12.

Both are equivalent

Best Regards,
Georgi

[pietinger]

pietinger
Mine is a file, by choice, even in my most recent install. I find it much easier to edit a single file with the entries in alphabetic order, especially since sometimes entries are closely related, as when trying to migrate from one python version to another.

figueroa,
mine is also a file (even in my most recent install) I love it to see everything at once

You can still have a single file in the directory. However this gives you the opportunity to have other files, say you're trying something and you give the file a specific name to remind you of its purpose. After you're done with the trial, you can get rid of the file altogether, rather than having to edit your main file.

Yes, this is very true ... and your explanation is great also ... maybe this was the reason for introducing/creating this new option (which is default today).

[tckosvic]

thanks all again. I'm learning gentoo from the comments.

By the way, my emerge@world eventually finished satisfactorily (after about 24 hours) So, mission accomplished.