Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Sanity check, PYTHON_ TARGETS, SINGLE_TARGET: -python2_7
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17166

PostPosted: Wed Apr 18, 2018 11:50 pm    Post subject: Sanity check, PYTHON_ TARGETS, SINGLE_TARGET: -python2_7 Reply with quote

EDIT: I recommend starting with Ant. P's solution. I've switched to a modified version of that solution, but only for transition.



Until now, I have let the profile determine what the targets ought to be. I haven't been able to find much that addresses the dangers of manipulating the targets. In a general sense, I get it.

Project:Python/PYTHON TARGETS helped me guess the "-python2_7" usage.

From emerge --info:
Code:
PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5"
Code:
$ eselect python list
Available Python interpreters, in order of preference:
  [1]   python3.5
  [2]   python3.6 (fallback)
  [3]   python2.7 (fallback)


I was looking to install ansible. 21 new packages, 2 rebuilds. The rebuilds were for python 2.7 and 3.5 with USE sqlite. I wondered if rebuilding python 2.7 was really required; I'm fine with ansible using python 3.

After trial and error, I finally settled on the below package.use settings.

emerge -p ansible doesn't report any problems.

Is this a bad idea, and if so, how bad? I was thinking of expanding it to other installed apps that seem to be unnecessarily requiring python 2.7 (such as vim). Ideally I'd like to see the only apps using 2.7 be the apps that actually require it.

Code:
app-admin/ansible       PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-libs/libsodium-1.0.11      PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-libs/libev-4.23    PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=net-dns/c-ares-1.13.0  PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=net-libs/zeromq-4.2.2-r2       PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-lang/python-2.7.14-r1      PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/pyparsing-2.2.0     PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/greenlet-0.4.12     PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/httplib2-0.10.3-r1  PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/pyyaml-3.12         PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/decorator-4.2.1     PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/simplegeneric-0.8.1-r1      PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/ptyprocess-0.5.1    PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/py-1.4.34   PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/markupsafe-0.23     PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/jinja-2.10  PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/packaging-16.8      PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/pexpect-4.2.1       PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=virtual/python-greenlet-1.0    PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/gevent-1.2.1        PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/pynacl-1.1.2        PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/bcrypt-3.1.3        PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/pyzmq-16.0.2        PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/paramiko-2.4.1      PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/ipython-3.2.1-r1    PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=dev-python/netaddr-0.7.19      PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7
=net-misc/sshpass-1.05  PYTHON_TARGETS:  -python2_7 PYTHON_SINGLE_TARGET:  -python2_7

_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky


Last edited by pjp on Sat May 19, 2018 10:17 pm; edited 1 time in total
Back to top
View user's profile Send private message
fcl
n00b
n00b


Joined: 31 Dec 2016
Posts: 71

PostPosted: Thu Apr 19, 2018 5:00 am    Post subject: Reply with quote

Why not go with a whitelist instead?
Code:
*/* PYTHON_TARGETS: -* python3_5
*/* PYTHON_SINGLE_TARGET: -* python3_5

Then add whitelisted 2_7 items as needed, eg.
Code:
app-text/asciidoc python_targets_python2_7 python_single_target_python2_7


edit: the later single_target one can be omitted in most cases.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17166

PostPosted: Thu Apr 19, 2018 3:20 pm    Post subject: Reply with quote

"-* python3_5" forces only 3_5, doesn't it?

My thinking with -python2_7 was that other choices would be addressed by Portage.

I guess I'm just uncertain down which path lies more effort to maintain. And it shouldn't be too difficult to switch or undo.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
Ant P.
Advocate
Advocate


Joined: 18 Apr 2009
Posts: 4984

PostPosted: Thu Apr 19, 2018 4:31 pm    Post subject: Reply with quote

I've been doing something slightly more extreme - whitelisting all use of python. I started doing this after the 3.4->3.5 transition because I saw most of the changes were in packages with USE="-python" on them already, thus useless. If I'm going to be forced to rebuild them all anyway, might as well make sure it doesn't happen again.

Here's the package.use (it used to be much longer, but I uninstalled chromium):
Code:
# Disable python entirely by default
*/* -python PYTHON_TARGETS: -* PYTHON_SINGLE_TARGET: -*

# Things that need Python 2 because something that depends on them doesn't support 3
#net-wireless/crda-3.18-r1 requires:
dev-python/m2crypto            PYTHON_TARGETS: python2_7
dev-python/setuptools          PYTHON_TARGETS: python2_7
dev-python/typing              PYTHON_TARGETS: python2_7

# Things that need Python but don't care which
app-misc/pax-utils             PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
app-portage/*                  PYTHON_TARGETS: python3_6
dev-java/java-config           PYTHON_TARGETS: python3_6
dev-python/*                   PYTHON_TARGETS: python3_6
dev-util/gdbus-codegen         PYTHON_TARGETS: python3_6
dev-util/meson                 PYTHON_TARGETS: python3_6
dev-vcs/src                    PYTHON_TARGETS: python3_6
kde-frameworks/ki18n           PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
media-gfx/blender              PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
media-video/mpv                PYTHON_TARGETS: python3_6
net-misc/youtube-dl            PYTHON_TARGETS: python3_6
sys-apps/portage               PYTHON_TARGETS: python3_6
sys-process/htop               PYTHON_TARGETS: python3_6 PYTHON_SINGLE_TARGET: python3_6
virtual/python-ipaddress       PYTHON_TARGETS: python3_6
virtual/python-pathlib         PYTHON_TARGETS: python3_6
virtual/python-singledispatch  PYTHON_TARGETS: python3_6
x11-proto/*                    PYTHON_TARGETS: python3_6

# Stupid ebuilds that have a hard Python 2 requirement yet expose a USE flag for it, which we have to re-enable
app-text/asciidoc              PYTHON_TARGETS: python2_7 PYTHON_SINGLE_TARGET: python2_7
dev-java/javatoolkit           PYTHON_TARGETS: python2_7
dev-util/scons                 PYTHON_TARGETS: python2_7
dev-util/systemtap             PYTHON_TARGETS: python2_7
gnome-base/gconf               PYTHON_TARGETS: python2_7
net-proxy/http-replicator      PYTHON_TARGETS: python2_7
sys-devel/distcc               PYTHON_TARGETS: python2_7

_________________
*.ebuild // /etc/service/*
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17166

PostPosted: Thu Apr 19, 2018 10:51 pm    Post subject: Reply with quote

Hmm. Thanks. At least I'm not doing something horribly scary :)

I'm guessing Portage will complain if something needs, in your example, a python newer than 3_6?

What about knowing when newer targets become available? Do you just grep in profiles?


Thanks to both of you for the feedback.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 12240

PostPosted: Fri Apr 20, 2018 1:19 am    Post subject: Reply with quote

I had considered doing something like Ant P. describes, because I too noticed the silliness of a package that respects USE=-python to completely omit all Python support, but still wanting to rebuild to change to a different Python target that it would then not use, due to USE=-python. Fixing this would require some syntax extension in the ebuild so that the ebuild could declare that USE=-foo implies that USE=bar is ignored, so no rebuild is required to add/remove bar if foo is unset. Hypothetically, it could be expressed as:
Code:
# If USE=python is unset, python_targets_python2_7 is ignored, and can be changed freely without changing what the ebuild builds or installs.
IGNORED_USE="!python? ( python_targets_python2_7 )"
Assuming the maintainers set their dependencies correctly, yes, Portage should alert when local settings conflict with Python target requirements for a new/updated ebuild.

I expect that all Python targets, both enabled and disabled, both old and new, should be visible in emerge --verbose output for Python-aware packages. No grep should be required.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17166

PostPosted: Fri Apr 20, 2018 3:26 am    Post subject: Reply with quote

Maybe one of the developers is experiencing the same situation and has it on their shortlist :)

Thanks for the --verbose reminder. I normally didn't pay attention to the targets since I wasn't intervening. And it does seem to be there.

Code:
[ebuild   R    ] app-admin/ansible-2.4.3.0::gentoo  USE="{-test}" PYTHON_TARGETS="python3_5 -python2_7 -python3_6"


Thanks.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
Ant P.
Advocate
Advocate


Joined: 18 Apr 2009
Posts: 4984

PostPosted: Fri Apr 20, 2018 10:40 am    Post subject: Reply with quote

pjp wrote:
I'm guessing Portage will complain if something needs, in your example, a python newer than 3_6?

What about knowing when newer targets become available? Do you just grep in profiles?

I usually ignore python upgrades until a news item shows up announcing them... and then try to go one version higher than that to head off upgrades :wink:

Big fan of Hu's idea there. While we're at it, I wish the profile updates mechanism could contain things like foo/bar-1.2.3[use(+)] foo/bar-1.2.3[use(-)] to express "we've [added/deleted] this use flag, the [previous/new] state is equivalent to it being [set/unset], don't rebuild if it hasn't changed".
_________________
*.ebuild // /etc/service/*
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17166

PostPosted: Fri Apr 20, 2018 3:37 pm    Post subject: Reply with quote

Ant P. wrote:
I usually ignore python upgrades until a news item shows up announcing them... and then try to go one version higher than that to head off upgrades :wink:
:lol:

I may try that at some point. Thanks again. I think I at least have enough info to not make a mess :)
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17166

PostPosted: Sat May 19, 2018 10:13 pm    Post subject: Reply with quote

As I anticipated, my initial solution didn't take long before it became too brittle, so I've migrated to a modified / transitional form of Ant. P's solution.

Code:
"# Disable python entirely by default"
*/* -python PYTHON_TARGETS: -* PYTHON_SINGLE_TARGET: -*
This was a bit too aggressive for an initial transition, so added some entries to the file to avoid rebuilding a lot of stuff. I wanted a file that "didn't change what I had installed."

To prevent rebuilds due to the "-python" change, I gathered a list of packages that were going to be rebuilt without python support. I added them to a section of the same file.
Code:
# Enable python USE flag for packages already installed.
category/package        python


Changing PYTHON_TARGETS and PYTHON_SINGLE_TARGET to "-*" naturally wanted to rebuild stuff without python2_7 support. To avoid rebuilds for now, I then added a section to the same file.
Code:
# Already installed with 2_7 and 3_5
<category/package        PYTHON_TARGETS: python2_7 python3_5  PYTHON_SINGLE_TARGET: python3_5
<category/package        PYTHON_TARGETS: python2_7 python3_5


I haven't migrated to python 3.6, so I changed references from python3_6 to python3_5.

Over time I'll reduce what is installed with python2_7 and migrate to python3_6.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6017

PostPosted: Sun May 20, 2018 4:21 am    Post subject: Reply with quote

Hu wrote:
Code:
IGNORED_USE="!python? ( python_targets_python2_7 )"

This would be even an extension of runtime switchable USE-flags (because it would be conditionally runtime switchable USE-flags). The former was planned for EAPI=7 (and IIRC already previously for EAPI=6) but nobody ever implemented it in portage.
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 2802
Location: Dallas area

PostPosted: Sun May 20, 2018 10:33 am    Post subject: Reply with quote

In my case I wanted to block everything not python 2.7 while allowing one version of python 3 in, so in make.conf
PYTHON_TARGETS="python2_7"
PYTHON_SINGLE_TARGET="python2_7"

In package.mask block those python version(s) I don't ever want
dev-lang/python:3.3
dev-lang/python:3.4
dev-lang/python:3.5

In package.use these are the exceptions
dev-python/certifi python_targets_python3_6
dev-python/setuptools python_targets_python3_6
dev-util/meson python_targets_python3_6

I could just as easily have made python 3.6 the default while allowing a few python2 packages.
And either way there is some editing of package.use

Edit to add: and I ignore all requests to rebuild (no New use) because of changed "python" flags (whether adding or deleting)
as I figure that the next upgrade of whatever package will pick that up. :)
_________________
Asus m5a99fx, FX 8320 - amd64-multilib, 4.14.41-zen, glibc-2.25, gcc-6.4.0, eudev
xorg-server-1.19.5 w/mesa-17.2.8, openbox, nouveau and radeon, oss4(2017), profile 13.0 in local
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum