Python Scrapy 1.4

Troopo · Guru Joined: 14 Jun 2015 Posts: 310

Hi,

I need scrapy for a project it's a python lib, i've tried to install it via pip in the past and it messed up the entire system so now i want to do it via portage, AFAIK there was only 1 ebuild for v1.03 which is very old and that too dies a few months back.
I've found this one:
https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/scrapy/scrapy-1.4.0.ebuild?id=ad407789da9bde8519598415ce25a2a9706b70ae

and wanted to build it but it failed on 1 dependency dev-python/parsel which is connected to scrapy and is on the github sources.

Can anyone give me a hand here, i don't see any other choice apart from build it from github.

Anon-E-moose · Posted: Fri Dec 14, 2018 4:34 pm Post subject:

https://bugs.gentoo.org/626612

The 1.40 ebuild version of scrapy is here https://paste.pound-python.org/show/br42I3sF6suizdHElUEs/
you would still need to download the source for both (I think that bsd repossitories still have both)

From reading the bugs, it should be easily moved up to 1.5.* with the same ebuild

Source
https://launchpad.net/ubuntu/+source/python-parsel/1.2.0-1 (rename to parsel-1.2.0.tar.gz)
https://launchpad.net/ubuntu/+source/python-scrapy/1.4.0-1 (rename to scrapy-1.4.0.tar.gz)
You do have to change the names of the *orig.tar.gz to what it expects in the ebuild, but it should work.

ETA: you also need queuelib (no longer in tree)
Source at http://ftp.debian.org/debian/pool/main/p/python-queuelib/python-queuelib_1.1.1.orig.tar.gz (rename to queuelib-1.1.1.tar.gz )
Ebuild https://paste.pound-python.org/show/SQ0SSVqjbYSlbAUwmu5i/

ETA2: looks like several things aren't in portage, you'll need to chase them down, use google and the file name that the ebuild expects.

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9fb89deb17260beb80c2d06de6f6a9f02226c294 - w3lib tar.gz rename
https://paste.pound-python.org/show/b3Vhj3K5IsZHMmt2JpWI/ (ebuild for e3lib 1.18.0)

With these they at least try and emerge. good luck
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland