View previous topic :: View next topic |
Author |
Message |
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Wed Apr 18, 2018 11:50 pm Post subject: Sanity check, PYTHON_ TARGETS, SINGLE_TARGET: -python2_7 |
|
|
EDIT: I recommend starting with Ant. P's solution. I've switched to a modified version of that solution, but only for transition.
Until now, I have let the profile determine what the targets ought to be. I haven't been able to find much that addresses the dangers of manipulating the targets. In a general sense, I get it.
Project:Python/PYTHON TARGETS helped me guess the "-python2_7" usage.
From emerge --info: Code: | PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5" | Code: | $ eselect python list
Available Python interpreters, in order of preference:
[1] python3.5
[2] python3.6 (fallback)
[3] python2.7 (fallback) |
I was looking to install ansible. 21 new packages, 2 rebuilds. The rebuilds were for python 2.7 and 3.5 with USE sqlite. I wondered if rebuilding python 2.7 was really required; I'm fine with ansible using python 3.
After trial and error, I finally settled on the below package.use settings.
emerge -p ansible doesn't report any problems.
Is this a bad idea, and if so, how bad? I was thinking of expanding it to other installed apps that seem to be unnecessarily requiring python 2.7 (such as vim). Ideally I'd like to see the only apps using 2.7 be the apps that actually require it.
Code: | app-admin/ansible PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-libs/libsodium-1.0.11 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-libs/libev-4.23 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=net-dns/c-ares-1.13.0 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=net-libs/zeromq-4.2.2-r2 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-lang/python-2.7.14-r1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/pyparsing-2.2.0 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/greenlet-0.4.12 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/httplib2-0.10.3-r1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/pyyaml-3.12 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/decorator-4.2.1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/simplegeneric-0.8.1-r1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/ptyprocess-0.5.1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/py-1.4.34 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/markupsafe-0.23 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/jinja-2.10 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/packaging-16.8 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/pexpect-4.2.1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=virtual/python-greenlet-1.0 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/gevent-1.2.1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/pynacl-1.1.2 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/bcrypt-3.1.3 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/pyzmq-16.0.2 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/paramiko-2.4.1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/ipython-3.2.1-r1 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=dev-python/netaddr-0.7.19 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7
=net-misc/sshpass-1.05 PYTHON_TARGETS: -python2_7 PYTHON_SINGLE_TARGET: -python2_7 |
_________________ Quis separabit? Quo animo?
Last edited by pjp on Sat May 19, 2018 10:17 pm; edited 1 time in total |
|
Back to top |
|
|
fcl Tux's lil' helper
Joined: 31 Dec 2016 Posts: 77
|
Posted: Thu Apr 19, 2018 5:00 am Post subject: |
|
|
Why not go with a whitelist instead?
Code: | */* PYTHON_TARGETS: -* python3_5
*/* PYTHON_SINGLE_TARGET: -* python3_5 |
Then add whitelisted 2_7 items as needed, eg.
Code: | app-text/asciidoc python_targets_python2_7 python_single_target_python2_7 |
edit: the later single_target one can be omitted in most cases. |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Thu Apr 19, 2018 3:20 pm Post subject: |
|
|
"-* python3_5" forces only 3_5, doesn't it?
My thinking with -python2_7 was that other choices would be addressed by Portage.
I guess I'm just uncertain down which path lies more effort to maintain. And it shouldn't be too difficult to switch or undo. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Thu Apr 19, 2018 4:31 pm Post subject: |
|
|
I've been doing something slightly more extreme - whitelisting all use of python. I started doing this after the 3.4->3.5 transition because I saw most of the changes were in packages with USE="-python" on them already, thus useless. If I'm going to be forced to rebuild them all anyway, might as well make sure it doesn't happen again.
Here's the package.use (it used to be much longer, but I uninstalled chromium):
Code: | # Disable python entirely by default
*/* -python PYTHON_TARGETS: -* PYTHON_SINGLE_TARGET: -*
# Things that need Python 2 because something that depends on them doesn't support 3
#net-wireless/crda-3.18-r1 requires:
dev-python/m2crypto PYTHON_TARGETS: python2_7
dev-python/setuptools PYTHON_TARGETS: python2_7
dev-python/typing PYTHON_TARGETS: python2_7
# Things that need Python but don't care which
app-misc/pax-utils PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
app-portage/* PYTHON_TARGETS: python3_6
dev-java/java-config PYTHON_TARGETS: python3_6
dev-python/* PYTHON_TARGETS: python3_6
dev-util/gdbus-codegen PYTHON_TARGETS: python3_6
dev-util/meson PYTHON_TARGETS: python3_6
dev-vcs/src PYTHON_TARGETS: python3_6
kde-frameworks/ki18n PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
media-gfx/blender PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
media-video/mpv PYTHON_TARGETS: python3_6
net-misc/youtube-dl PYTHON_TARGETS: python3_6
sys-apps/portage PYTHON_TARGETS: python3_6
sys-process/htop PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
virtual/python-ipaddress PYTHON_TARGETS: python3_6
virtual/python-pathlib PYTHON_TARGETS: python3_6
virtual/python-singledispatch PYTHON_TARGETS: python3_6
x11-proto/* PYTHON_TARGETS: python3_6
# Stupid ebuilds that have a hard Python 2 requirement yet expose a USE flag for it, which we have to re-enable
app-text/asciidoc PYTHON_TARGETS: python2_7 PYTHON_SINGLE_TARGET: python2_7
dev-java/javatoolkit PYTHON_TARGETS: python2_7
dev-util/scons PYTHON_TARGETS: python2_7
dev-util/systemtap PYTHON_TARGETS: python2_7
gnome-base/gconf PYTHON_TARGETS: python2_7
net-proxy/http-replicator PYTHON_TARGETS: python2_7
sys-devel/distcc PYTHON_TARGETS: python2_7 |
|
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Thu Apr 19, 2018 10:51 pm Post subject: |
|
|
Hmm. Thanks. At least I'm not doing something horribly scary :)
I'm guessing Portage will complain if something needs, in your example, a python newer than 3_6?
What about knowing when newer targets become available? Do you just grep in profiles?
Thanks to both of you for the feedback. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21635
|
Posted: Fri Apr 20, 2018 1:19 am Post subject: |
|
|
I had considered doing something like Ant P. describes, because I too noticed the silliness of a package that respects USE=-python to completely omit all Python support, but still wanting to rebuild to change to a different Python target that it would then not use, due to USE=-python. Fixing this would require some syntax extension in the ebuild so that the ebuild could declare that USE=-foo implies that USE=bar is ignored, so no rebuild is required to add/remove bar if foo is unset. Hypothetically, it could be expressed as: Code: | # If USE=python is unset, python_targets_python2_7 is ignored, and can be changed freely without changing what the ebuild builds or installs.
IGNORED_USE="!python? ( python_targets_python2_7 )" | Assuming the maintainers set their dependencies correctly, yes, Portage should alert when local settings conflict with Python target requirements for a new/updated ebuild.
I expect that all Python targets, both enabled and disabled, both old and new, should be visible in emerge --verbose output for Python-aware packages. No grep should be required. |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Fri Apr 20, 2018 3:26 am Post subject: |
|
|
Maybe one of the developers is experiencing the same situation and has it on their shortlist :)
Thanks for the --verbose reminder. I normally didn't pay attention to the targets since I wasn't intervening. And it does seem to be there.
Code: | [ebuild R ] app-admin/ansible-2.4.3.0::gentoo USE="{-test}" PYTHON_TARGETS="python3_5 -python2_7 -python3_6" |
Thanks. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
Ant P. Watchman
Joined: 18 Apr 2009 Posts: 6920
|
Posted: Fri Apr 20, 2018 10:40 am Post subject: |
|
|
pjp wrote: | I'm guessing Portage will complain if something needs, in your example, a python newer than 3_6?
What about knowing when newer targets become available? Do you just grep in profiles? |
I usually ignore python upgrades until a news item shows up announcing them... and then try to go one version higher than that to head off upgrades
Big fan of Hu's idea there. While we're at it, I wish the profile updates mechanism could contain things like foo/bar-1.2.3[use(+)] foo/bar-1.2.3[use(-)] to express "we've [added/deleted] this use flag, the [previous/new] state is equivalent to it being [set/unset], don't rebuild if it hasn't changed". |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Fri Apr 20, 2018 3:37 pm Post subject: |
|
|
Ant P. wrote: | I usually ignore python upgrades until a news item shows up announcing them... and then try to go one version higher than that to head off upgrades |
I may try that at some point. Thanks again. I think I at least have enough info to not make a mess _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20067
|
Posted: Sat May 19, 2018 10:13 pm Post subject: |
|
|
As I anticipated, my initial solution didn't take long before it became too brittle, so I've migrated to a modified / transitional form of Ant. P's solution.
Code: | "# Disable python entirely by default"
*/* -python PYTHON_TARGETS: -* PYTHON_SINGLE_TARGET: -* | This was a bit too aggressive for an initial transition, so added some entries to the file to avoid rebuilding a lot of stuff. I wanted a file that "didn't change what I had installed."
To prevent rebuilds due to the "-python" change, I gathered a list of packages that were going to be rebuilt without python support. I added them to a section of the same file. Code: | # Enable python USE flag for packages already installed.
category/package python |
Changing PYTHON_TARGETS and PYTHON_SINGLE_TARGET to "-*" naturally wanted to rebuild stuff without python2_7 support. To avoid rebuilds for now, I then added a section to the same file. Code: | # Already installed with 2_7 and 3_5
<category/package PYTHON_TARGETS: python2_7 python3_5 PYTHON_SINGLE_TARGET: python3_5
<category/package PYTHON_TARGETS: python2_7 python3_5 |
I haven't migrated to python 3.6, so I changed references from python3_6 to python3_5.
Over time I'll reduce what is installed with python2_7 and migrate to python3_6. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6747
|
Posted: Sun May 20, 2018 4:21 am Post subject: |
|
|
Hu wrote: | Code: | IGNORED_USE="!python? ( python_targets_python2_7 )" |
|
This would be even an extension of runtime switchable USE-flags (because it would be conditionally runtime switchable USE-flags). The former was planned for EAPI=7 (and IIRC already previously for EAPI=6) but nobody ever implemented it in portage. |
|
Back to top |
|
|
Anon-E-moose Watchman
Joined: 23 May 2008 Posts: 6098 Location: Dallas area
|
Posted: Sun May 20, 2018 10:33 am Post subject: |
|
|
In my case I wanted to block everything not python 2.7 while allowing one version of python 3 in, so in make.conf
PYTHON_TARGETS="python2_7"
PYTHON_SINGLE_TARGET="python2_7"
In package.mask block those python version(s) I don't ever want
dev-lang/python:3.3
dev-lang/python:3.4
dev-lang/python:3.5
In package.use these are the exceptions
dev-python/certifi python_targets_python3_6
dev-python/setuptools python_targets_python3_6
dev-util/meson python_targets_python3_6
I could just as easily have made python 3.6 the default while allowing a few python2 packages.
And either way there is some editing of package.use
Edit to add: and I ignore all requests to rebuild (no New use) because of changed "python" flags (whether adding or deleting)
as I figure that the next upgrade of whatever package will pick that up. _________________ PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
Back to top |
|
|
|