View previous topic :: View next topic |
Author |
Message |
teika Apprentice
Joined: 19 Feb 2011 Posts: 155 Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
Posted: Mon Apr 02, 2012 1:02 pm Post subject: Python-2/3 vs pypy (not recommendable) as "emerge" |
|
|
In this thread, the performances of python-3 (and 2) and pypy are compared to run "emerge".
Conclusion: Stick to python-3, although it seems to vary greatly among environments. There doesn't seem to exist any special reason to use pypy(-bin), at least for pypy-1.9 and 2.0, although it's often said that pypy is far faster than Cpython.
Edit history
* Apr 2012: portage can use pypy as a backend.
* 1 Aug 2013: pypy-bin is in portage tree.
* 18 Aug 2013: performance comparison data have accumulated.
How to use pypy(-bin)-1.9 as the portage backend:
0. (pypy(-bin):1.9 is now masked. pypy:2.0 is available, but pypy-bin-2.0.2 can't run portage.)
1. Add
sys-apps/portage -pypy1_9
to /etc/portage/profile/package.use.mask (the preceding hyphen in "-pypy1_9" is correct here.) to unmask a USE flag.
2. Add
sys-apps/portage pypy1_9
to /etc/portage/package.use.
3. Add
dev-python/pypy-bin:1.9
virtual/pypy:1.9
to /etc/portage/package.keywords
4. Re-emerge sys-apps/portage, and you're done!
Notes:
pypy vs pypy-bin
Building of pypy takes long; according to my slack observation, it seems some 13 times of building a kernel. But you can avoid building by emerging dev-python/pypy-bin:1.9, the binary package of pypy.
pypy-bin: 1.9 vs 2.0
As for the pypy-bin, you can't install sys-apps/portage with =dev-python/pypy-bin-2.0.2, but =dev-python/pypy-bin-1.9 works. With 2.0.2, the sanity check fails:
build log wrote: | >>> Merging sys-apps/portage-2.1.12.2 to /
* Running preinst sanity tests...
testBashSyntax (portage.tests.lint.test_bash_syntax.BashSyntaxTestCase) ... ok
testCompileModules (portage.tests.lint.test_compile_modules.CompileModulesTestCa se) ... ok
testImportModules (portage.tests.lint.test_import_modules.ImportModulesTestCase) ... FAIL |
I only tested pypy-bin-2.0.2. From what we gather here, a home-built pypy-2.0.2 with appropriate USE flags seems ok for portage.
Unmasking a USE flag
Currently, the USE flag pypy1_9 is masked. You can unmask it by preparing /etc/portage/profile/package.use.mask. User "cgw" said below to use /etc/portage/make.profile/..., but /etc/portage/make.profile is a symlink to somewhere under /usr/portage/profile, and you shouldn't use it. /etc/portage/profile is for your personal configuration.
If you also want to use pypy(-bin) for packages other than sys-apps/portage, then you have to unmask another USE flag, "python_targets_pypy1_9".
If you screwed up things, you can install the portage binary from the tinderbox. default-linux/<your arch> is a moderate choice.
Thanks all. _________________ Hack of easy Shift / Ctrl / AltGr etc; save your pinkies, type without drudgery: topic 865313
XPAT - Xi, Putin, Abe and Trump - are security holes of their own nations.
Last edited by teika on Sun Aug 18, 2013 11:25 am; edited 4 times in total |
|
Back to top |
|
|
cgw n00b
Joined: 04 Oct 2012 Posts: 3
|
Posted: Thu Oct 04, 2012 4:40 pm Post subject: Is anyone using pypy for portage? |
|
|
Is anyone using pypy for portage? I tried "USE=-python2 -python3 pypy1_9" and /usr/bin/emerge still runs with cpython2.7, not pypy. I created (possibly prematurely?) a bug report for this, containing more detail.
https://bugs.gentoo.org/show_bug.cgi?id=437198 |
|
Back to top |
|
|
cgw n00b
Joined: 04 Oct 2012 Posts: 3
|
Posted: Thu Oct 04, 2012 5:56 pm Post subject: fixed! |
|
|
Turns out the USE flag was masked (in /usr/portage/profiles/package.use.mask). Adding
a file /etc/portage/profile/package.use.mask/portage.use.mask containing the line
sys-apps/portage -pypy1_9
is sufficient to unmask the pypy1_9 USE flag.
(Clearly, I didn't know about masked USE flags, with Gentoo there's always something new to learn!) |
|
Back to top |
|
|
one_and_only Apprentice
Joined: 13 May 2007 Posts: 250 Location: PL/Krakow
|
Posted: Thu Nov 01, 2012 12:09 pm Post subject: |
|
|
anybody using/hacking pypy backend?
edit: quick benchmark of emerge -auND @world for quite not up-to-date system (time of resolving update graph, not the whole emerging):
Code: |
pypy:
emerge --update --newuse --keep-going --deep --tree --quiet-build=y -a @worl 168.28s user 5.95s system 99% cpu 2:55.16 total
python3:
emerge --update --newuse --keep-going --deep --tree --quiet-build=y -a @worl 271.40s user 6.33s system 99% cpu 4:38.64 total
python2:
emerge --update --newuse --keep-going --deep --tree --quiet-build=y -a @worl 323.13s user 6.50s system 99% cpu 5:30.59 total
|
|
|
Back to top |
|
|
teika Apprentice
Joined: 19 Feb 2011 Posts: 155 Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
Posted: Thu Aug 01, 2013 5:24 am Post subject: portage w/ pypy-bin howto |
|
|
Now pypy-bin is in the portage tree. I've posted a howto, by rewriting the first post of this thread. Thanks cgw for the howto of unmasking USE flags. |
|
Back to top |
|
|
teika Apprentice
Joined: 19 Feb 2011 Posts: 155 Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
Posted: Fri Aug 09, 2013 2:11 am Post subject: |
|
|
Yesterday, pypy-bin:1.9 was masked. =P |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6747
|
Posted: Fri Aug 09, 2013 7:37 am Post subject: |
|
|
Building pypy requires the insane amount of 2GB RAM. Before I make my system slower or even useless: Can anybody tell me how much RAM is required with pypy-2.0 for running portage, e.g. Code: | /usr/bin/time -f '%M kB' emerge -peD @world | with perhaps 1500 installed packages? |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6051 Location: Removed by Neddy
|
Posted: Fri Aug 09, 2013 7:29 pm Post subject: |
|
|
shouldn't this be in doc's and tips? _________________
Quote: | Removed by Chiitoo |
|
|
Back to top |
|
|
Civil Retired Dev
Joined: 24 Feb 2006 Posts: 16 Location: Berlin
|
Posted: Wed Aug 14, 2013 6:16 pm Post subject: |
|
|
1413 packages. Portage 2.2.0:
pypy2_0:
Code: | 77.91user 1.91system 1:19.79elapsed 100%CPU (0avgtext+0avgdata 3131264maxresident)k
0inputs+48outputs (0major+451657minor)pagefaults 0swap |
python3:
Code: | 49.80user 1.51system 0:51.25elapsed 100%CPU (0avgtext+0avgdata 1467344maxresident)k
0inputs+48outputs (0major+368221minor)pagefaults 0swaps |
python2.7:
Code: | 57.23user 1.68system 0:58.87elapsed 100%CPU (0avgtext+0avgdata 1416000maxresident)k
0inputs+48outputs (0major+364283minor)pagefaults 0swaps |
Conclusion: Twice ram, 30% slower.
HW: Core i7-3517U (2 cores, 1.9GHz), 4GB Ram, SSD. _________________ ‘Who controls the past controls the future: who controls the present controls the past.’
George Orwell ‘1984’ |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6747
|
Posted: Wed Aug 14, 2013 7:54 pm Post subject: |
|
|
Civil wrote: | Conclusion: Twice ram, 30% slower.
HW: Core i7-3517U (2 cores, 1.9GHz), 4GB Ram, SSD. |
Yes, it seems with <4GB one better should not attempt to use it. Thanks. |
|
Back to top |
|
|
derlafff n00b
Joined: 15 Aug 2013 Posts: 1
|
Posted: Thu Aug 15, 2013 9:49 am Post subject: |
|
|
I tested performance of -peD with 780 packages installed:
Code: | python3.2:
real 0m15.112s
user 0m14.817s
sys 0m0.219s
pypy2.0:
real 0m31.482s
user 0m30.619s
sys 0m0.352s
|
Intel Core i5-3230M, 8Gb RAM, HDD
pypy is 2 times slower, python2.7 is a little bit slower than python 3.2 |
|
Back to top |
|
|
teika Apprentice
Joined: 19 Feb 2011 Posts: 155 Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
Posted: Sun Aug 18, 2013 11:00 am Post subject: |
|
|
Hmm, it seems to greatly depend. Conclusion for me is: they scarcely differ. The output of "/usr/bin/time emerge -peD":
Code: | pypy-bin-1.9:
119.26user 2.99system 2:58.14elapsed 68%CPU (0avgtext+0avgdata 611440maxresident)k
0inputs+0outputs (1major+46678minor)pagefaults 0swaps
(drop cache)
python-3.2:
119.56user 3.22system 3:02.41elapsed 67%CPU (0avgtext+0avgdata 611200maxresident)k
0inputs+0outputs (51major+46466minor)pagefaults 0swaps |
Single core amd64, 615 packages. (Very slow machine. ;-)
Maybe pypy-2.0 is significanly slower and uses more RAM than pypy-1.9 ? |
|
Back to top |
|
|
Civil Retired Dev
Joined: 24 Feb 2006 Posts: 16 Location: Berlin
|
Posted: Tue Aug 20, 2013 12:21 am Post subject: |
|
|
I've tested on desktop (Core i7-4770@4.4GHz, 16GB Ram, HDD):
1510 pkgs
python2.7:
Code: | real 1m20.119s
user 1m19.284s
sys 0m0.825s
78.53user 0.82system 1:19.36elapsed 99%CPU (0avgtext+0avgdata 1817632maxresident)k
0inputs+0outputs (0major+127789minor)pagefaults 0swaps |
python3:
Code: | real 1m6.659s
user 1m5.835s
sys 0m0.813s
65.79user 0.81system 1:06.63elapsed 99%CPU (0avgtext+0avgdata 1876432maxresident)k
200inputs+0outputs (1major+130936minor)pagefaults 0swaps |
pypy2_0:
Code: | real 1m1.223s
user 1m0.134s
sys 0m1.078s
59.16user 1.02system 1:00.19elapsed 99%CPU (0avgtext+0avgdata 4344896maxresident)k
0inputs+192outputs (0major+303639minor)pagefaults 0swaps |
_________________ ‘Who controls the past controls the future: who controls the present controls the past.’
George Orwell ‘1984’ |
|
Back to top |
|
|
|