| View previous topic :: View next topic |
| Author |
Message |
tkzv
Tux's lil' helper
Joined: 22 Aug 2014
Posts: 98
|
 Posted: Wed Mar 15, 2023 7:49 pm Post subject: pip refuses to work in externally-managed-environment |
 |
|
When I try to update a python package with pip, I get an error:
| Code: | $ pip3 install gallery-dl --upgrade --user
error: externally-managed-environment
× This environment is externally managed
╰─>
The system-wide Python installation in Gentoo should be maintained
using the system package manager (e.g. emerge).
If the package in question is not packaged for Gentoo, please
consider installing it inside a virtual environment, e.g.:
python -m venv /path/to/venv
. /path/to/venv/bin/activate
pip install mypackage
To exit the virtual environment, run:
deactivate
The virtual environment is not deleted, and can be re-entered by
re-sourcing the activate file.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
|
Since this package changes frequently, following changes at the sites it downloads, I prefer to use pip, since Portage takes longer to update ebuilds.
Is there a way to revert to the old behaviour? |
|
| Back to top |
|
 |
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23361
|
| Posted: Wed Mar 15, 2023 8:16 pm Post subject: |
|
|
| I do not see that error message in my copy of pip. What is the output of emerge --info; emerge --pretend --verbose dev-lang/python dev-python/pip; type python pip3; python --version; pip3 --version; python3 -c 'import pip;print(pip);'? |
|
| Back to top |
|
|
tkzv
Tux's lil' helper
Joined: 22 Aug 2014
Posts: 98
|
| Posted: Wed Mar 15, 2023 9:47 pm Post subject: |
|
|
| Hu wrote: | | I do not see that error message in my copy of pip. What is the output of emerge --info; emerge --pretend --verbose dev-lang/python dev-python/pip; type python pip3; python --version; pip3 --version; python3 -c 'import pip;print(pip);'? |
| Code: | LC_ALL='C' emerge --info; LC_ALL='C' emerge --pretend --verbose dev-lang/python dev-python/pip; LC_ALL='C' type python pip3; LC_ALL='C' python --version; LC_ALL='C' pip3 --version; LC_ALL='C' python3 -c 'import pip;print(pip);'
Portage 3.0.44 (python 3.10.10-final-0, default/linux/amd64/17.1/desktop/plasma, gcc-11, glibc-2.36-r7, 5.15.32-gentoo-r1 x86_64)
=================================================================
System uname: Linux-5.15.32-gentoo-r1-x86_64-AMD_FX-tm-8300_Eight-Core_Processor-with-glibc2.36
KiB Mem: 32933112 total, 3924328 free
KiB Swap: 65427452 total, 60872796 free
Timestamp of repository gentoo: Wed, 15 Mar 2023 18:45:01 +0000
Head commit of repository gentoo: a9a70b79634b07b6d6139cf2ab4bc9e39bb1e9f8
Timestamp of repository torbrowser: Mon, 13 Mar 2023 21:57:24 +0000
Head commit of repository torbrowser: 59b99b1f0d4c5d525514fb13837029e47736e4eb
sh bash 5.1_p16-r2
ld GNU ld (Gentoo 2.37_p1 p2) 2.37
app-misc/pax-utils: 1.3.5::gentoo
app-shells/bash: 5.1_p16-r2::gentoo
dev-java/java-config: 2.3.1::gentoo
dev-lang/perl: 5.36.0-r1::gentoo
dev-lang/python: 2.7.18_p16-r1::gentoo, 3.8.16_p4::gentoo, 3.9.16_p3::gentoo, 3.10.10_p3::gentoo, 3.11.2_p2::gentoo
dev-lang/rust: 1.66.1::gentoo
dev-lang/rust-bin: 1.62.1::gentoo
dev-util/cmake: 3.25.2::gentoo
dev-util/meson: 1.0.1::gentoo
sys-apps/baselayout: 2.13-r1::gentoo
sys-apps/openrc: 0.45.2-r2::gentoo
sys-apps/sandbox: 2.29::gentoo
sys-devel/autoconf: 2.13-r7::gentoo, 2.71-r5::gentoo
sys-devel/automake: 1.16.5::gentoo
sys-devel/binutils: 2.37_p1-r2::gentoo, 2.38-r2::gentoo, 2.39-r4::gentoo
sys-devel/binutils-config: 5.4.1::gentoo
sys-devel/clang: 13.0.1::gentoo, 14.0.6-r1::gentoo, 15.0.7-r1::gentoo
sys-devel/gcc: 11.3.1_p20221209::gentoo, 12.2.1_p20230121-r1::gentoo
sys-devel/gcc-config: 2.8::gentoo
sys-devel/libtool: 2.4.7-r1::gentoo
sys-devel/lld: 14.0.6-r1::gentoo
sys-devel/llvm: 13.0.1::gentoo, 14.0.6-r2::gentoo, 15.0.7::gentoo
sys-devel/make: 4.3::gentoo
sys-kernel/linux-headers: 6.1::gentoo (virtual/os-headers)
sys-libs/glibc: 2.36-r7::gentoo
Repositories:
gentoo
location: /usr/portage
sync-type: rsync
sync-uri: rsync://rsync.ru.gentoo.org/gentoo-portage
priority: -1000
volatile: True
sync-rsync-extra-opts:
sync-rsync-verify-metamanifest: yes
sync-rsync-verify-jobs: 1
sync-rsync-verify-max-age: 24
local_overlay
location: /usr/local/portage
masters: gentoo
priority: 0
volatile: True
torbrowser
location: /var/db/repos/torbrowser
sync-type: git
sync-uri: https://github.com/gentoo-mirror/torbrowser.git
masters: gentoo
volatile: True
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://mirror.yandex.ru/gentoo-distfiles/ ftp://mirror.netcologne.de/gentoo/ rsync://mirror.neolabs.kz/gentoo ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.snt.utwente.nl/pub/os/linux/gentoo rsync://de-mirror.org/gentoo/ ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ ftp://ftp.free.fr/mirrors/ftp.gentoo.org/ rsync://mirror.netcologne.de/gentoo/ http://de-mirror.org/gentoo/ ftp://de-mirror.org/gentoo/ ftp://mirror.yandex.ru/gentoo-distfiles/"
LANG="ru_RU.utf8"
LC_ALL="C"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LEX="flex"
LINGUAS="en ru ja zh_CN zh_TW"
MAKEOPTS="-j9"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi activities alsa amd64 bluetooth branding bzip2 cairo cdda cdr cli crypt dbus declarative dri dts dvd dvdr elogind encode exif ffmpeg flac fortran gdbm gif gpm gtk gui iconv icu idn ipv6 jbig jpeg jpeg2k kde kwallet lcms libglvnd libnotify libtirpc lzma mad mmx mng mp3 mp4 mpeg multilib ncurses nls nptl ogg old-kernel opengl openmp pam pango pcre pdf plasma png pnm policykit ppds qml qt3support qt5 readline sdl seccomp sound spell split-usr sqlite sse sse2 ssl startup-notification svg test-rust threads tiff truetype udev udisks unicode upower usb vorbis widgets wxwidgets x264 xcb xft xml xv xvid zlib" ABI_X86="64 32" ADA_TARGET="gnat_2021" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx f16c fma3 fma4 mmx mmxext pclmul popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-GB ru ja zh-CN zh-TW" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-4 php8-0" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_10" PYTHON_TARGETS="python3_8 python3_9 python3_10" RUBY_TARGETS="ruby27 ruby30" SANE_BACKENDS="genesys" USERLAND="GNU" VIDEO_CARDS="radeon r600 amdgpu radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
These are the packages that would be merged, in order:
Calculating dependencies... done!
Dependency resolution took 8.53 s.
[ebuild R ] dev-python/pip-23.0.1::gentoo USE="(test-rust) -test" PYTHON_TARGETS="python3_9 python3_10 (-pypy3) -python3_11" 0 KiB
[ebuild R ] dev-lang/python-3.11.2_p2:3.11::gentoo USE="ensurepip gdbm ncurses readline sqlite ssl tk -bluetooth -build -examples -hardened -libedit -lto -pgo -test -valgrind -verify-sig" 0 KiB
Total: 2 packages (2 reinstalls), Size of downloads: 0 KiB
python is hashed (/usr/bin/python)
pip3 is hashed (/usr/bin/pip3)
Python 3.10.10
pip 23.0.1 from /usr/lib/python3.10/site-packages/pip (python 3.10)
<module 'pip' from '/usr/lib/python3.10/site-packages/pip/__init__.py'>
|
|
|
| Back to top |
|
|
tkzv
Tux's lil' helper
Joined: 22 Aug 2014
Posts: 98
|
| Posted: Wed Mar 15, 2023 9:51 pm Post subject: |
|
|
| Code: | $ LC_ALL='C' eix -e dev-lang/python [I] dev-lang/python
Available versions:
(2.7) 2.7.18_p16-r1^t
(3.8) 3.8.16_p4^t
(3.9) 3.9.16_p3^t
(3.10) 3.10.10_p3^t
(3.11) 3.11.2_p2^t
(3.12) ~3.12.0_alpha6^t
{berkdb bluetooth build +ensurepip examples gdbm hardened libedit lto +ncurses pgo +readline +sqlite +ssl test tk valgrind verify-sig wininst +xml}
Installed versions: 2.7.18_p16-r1(2.7)^t(12:02:22 01/26/23)(gdbm ncurses readline sqlite ssl tk xml -berkdb -bluetooth -build -examples -hardened -valgrind -verify-sig -wininst) 3.8.16_p4(3.8)^t(00:15:19 03/16/23)(ensurepip gdbm ncurses readline sqlite ssl tk xml -bluetooth -build -examples -hardened -lto -pgo -test -valgrind -verify-sig -wininst) 3.9.16_p3(3.9)^t(00:08:05 03/16/23)(ensurepip gdbm ncurses readline sqlite ssl tk xml -bluetooth -build -examples -hardened -lto -pgo -test -valgrind -verify-sig) 3.10.10_p3(3.10)^t(00:04:29 03/16/23)(ensurepip gdbm ncurses readline sqlite ssl tk xml -bluetooth -build -examples -hardened -libedit -lto -pgo -test -valgrind -verify-sig) 3.11.2_p2(3.11)^t(00:11:37 03/16/23)(ensurepip gdbm ncurses readline sqlite ssl tk -bluetooth -build -examples -hardened -libedit -lto -pgo -test -valgrind -verify-sig)
Homepage: https://www.python.org/ https://github.com/python/cpython/
Description: An interpreted, interactive, object-oriented programming language
|
|
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23361
|
| Posted: Wed Mar 15, 2023 10:23 pm Post subject: |
|
|
| I was not familiar with this diagnostic before now. As I read the code, your options are to use a virtualenv as instructed, to use --target, or to bypass the safety check. I would have expected --user to avoid the diagnostic, but your experience suggests that does not work. |
|
| Back to top |
|
|
tkzv
Tux's lil' helper
Joined: 22 Aug 2014
Posts: 98
|
| Posted: Wed Mar 15, 2023 10:34 pm Post subject: |
|
|
| Hu wrote: | | I was not familiar with this diagnostic before now. As I read the code, your options are to use a virtualenv as instructed, to use --target, or to bypass the safety check. I would have expected --user to avoid the diagnostic, but your experience suggests that does not work. |
I wish to avoid the hassle with multiple virtual environments when I forget what is where. I want to update the programs already installed to the home directory. What target do I specify? What other opitions do I need?
gives an error "FileExistsError: [Errno 17] File exists: '/home/oleg/.local/share'":
| Code: | $ LC_ALL='C' pip3 install gallery-dl --upgrade --target ~/.local/
Collecting gallery-dl
Using cached gallery_dl-1.25.0-py3-none-any.whl (597 kB)
Collecting requests>=2.11.0
Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Collecting charset-normalizer<4,>=2
Using cached charset_normalizer-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Collecting idna<4,>=2.5
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting urllib3<1.27,>=1.21.1
Using cached urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, gallery-dl
Successfully installed certifi-2022.12.7 charset-normalizer-3.1.0 gallery-dl-1.25.0 idna-3.4 requests-2.28.2 urllib3-1.26.15
ERROR: Exception:
Traceback (most recent call last):
File "/usr/lib/python3.10/shutil.py", line 816, in move
os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/tmp/pip-target-jjous_wm/share' -> '/home/oleg/.local/share'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
status = run_func(*args)
File "/usr/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/usr/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 566, in run
self._handle_target_dir(
File "/usr/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 625, in _handle_target_dir
shutil.move(os.path.join(lib_dir, item), target_item_dir)
File "/usr/lib/python3.10/shutil.py", line 832, in move
copytree(src, real_dst, copy_function=copy_function,
File "/usr/lib/python3.10/shutil.py", line 559, in copytree
return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
File "/usr/lib/python3.10/shutil.py", line 457, in _copytree
os.makedirs(dst, exist_ok=dirs_exist_ok)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/home/oleg/.local/share'
|
|
|
| Back to top |
|
|
davidknippers
n00b
Joined: 07 Jun 2020
Posts: 5
|
| Posted: Thu Mar 16, 2023 7:29 pm Post subject: |
|
|
| i always used --user when installing from pip and it worked fine until my last sync/update. i'd really like this behaviour back. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23361
|
|
| Back to top |
|
|
Whoo
n00b
Joined: 25 May 2003
Posts: 27
Location: Quebec ;)
|
| Posted: Sun Mar 19, 2023 11:57 am Post subject: |
|
|
Hi,
Regarding discussion on forums the following works:
| Code: | | pip install requests --break-system-packages --user |
or
Adding this in ~/.config/pip/pip.conf
| Code: | [global]
break-system-packages = true |
Regards,
Whoo |
|
| Back to top |
|
|
jyoung
Guru
Joined: 20 Mar 2007
Posts: 485
|
| Posted: Wed Apr 05, 2023 3:16 pm Post subject: |
|
|
| I also ran into this issue this morning, and found this thread. I sounds like you folks have found the cause and mostly wrapped it up, but I'm wondering what best practices are going forward? I'm working with a python package called matplotlib. It's actually still possible to install this with emerge. |
|
| Back to top |
|
|
piii_man
n00b
Joined: 18 Jul 2004
Posts: 41
|
| Posted: Tue Apr 11, 2023 6:28 pm Post subject: |
|
|
| Quote: | | I'm wondering what best practices are going forward? |
The best practice is to use a virtual environment, like the error message says. It may seem cumbersome at first, but it can be streamlined by various higher level venv managers. You don't want the packages of a project to conflict with those of your OS or vice versa, and you don't want package versions of different python projects to conflict with each other. Here's a relevant link. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
