Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Automatically download torrent files using rsstorrent
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Fri Jan 27, 2012 11:14 am    Post subject: Automatically download torrent files using rsstorrent Reply with quote

Hi, here's a shameless plug for my own little utility =)

Summary:
I was lacking software that could automatically download torrent files based on a regular expression (or something like it) to my NAS (Excito B3). The torrent download is great, but it kind of defeats the purpose for me, since I still had to log in to my torrentsite, download the torrent, and manually put it on the torrent-folder on the B3.

Therefore I wrote a small program in python that monitors an RSS-feed and downloads all torrent files that match the files you have stated that you would like to download. It works for me on the B3, and you can either start it using screen or the init script that has recently been fixed (both Gentoo and Debian).

The config file is pretty self explanatory, but I have only the software with torrentbytes.net for the time being. I do hope that it works with whatever site you log-in to though.

If you have an account at a site, you have to get the RSS-feed from it (possibly with your hashed password and username in it), and you should then be ready to add it to the rsstorrent config-file and then be ready to roll!

Hope somebody else finds this useful. If you do, please throw me a few comments about what to improve, etc. The things that has been taking most of my development time is actually deciding on how the script should behave, and how the user should be able to control/configure/use it.

If you don't find it useful, please tell me why as well. Perhaps we can find space and time for adding some more features. But my idea is to keep it simple and have it do one thing pretty well.

I created a repository at gitHub, so either you clone it to your NAS/computer (to easily get fixes and changes) using git, or you can just download the files manually from the page.
https://github.com/AzP/rsstorrent

If you want to run it in a screen session (so it keeps on running after you've ended your session):

Code:
1. Log-in using SSH/start terminal.
2. Run 'screen -S rsstorrent' (use whatever name you want for the session)
3. Start the script (./rsstorrent.py or python rsstorrent.py)
4. Press ctrl+A ctrl+D


Now the session should be running in the background. Check it is by entering

Code:
screen -ls


That should show the name of your session. Reconnect to it the next time you SSH into the box by running

Code:
screen -r rsstorrent


2011-10-24 Big updates on the code this last week!

* Rewrote config handling to use ConfigParser
* Improved logging (now outputs to ~/.rsstorrent/rsstorrent.log as default or /var/log/rsstorrent/rsstorrent.log if run with init script)
* Fixed crash bugs caused by bad handling of non-resolved connection

Another solution, which would greatly simplify my rsstorrent-script, is to use a cron-job to launch it at specified intervals. That would mean that the code wouldn't have to handle being a daemon and run forever, but at the same time I wanted the code to be "self-contained" after install. Just install, edit your config-file, and then you're ready to go.

2012-01-27 Now with init scripts and proper Daemon mode!

Another "release" has been made as of yesterday. I've updated the script to run in proper Daemon mode, and use pid-files to keep track of itself. I also wrote the init script for the B3, but I forgot to commit it and push it to github, so sadly you can't get a hold of it right now. The Gentoo script is working though, it would be great to see people test it out!

This means that rsstorrent can now be added to automatically start at boot! It's definitely a big improvement in my point of view since I restart my B3/Desktop every now and then, and it's a pain to have to start a screen and run it inside that.

I'd really like some feedback from the Gentoo team regarding the init script, it's the first one I've written and I believe there are improvements to be made. I'm not entirely sure about the way I'm handling running it as a daemon.

    Features planned ahead:
  • Re-read (while running) the config file after it has been changed and supply log output regarding the new search patterns.

_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Tue Feb 07, 2012 3:25 pm    Post subject: Reply with quote

Updated version uploaded!

* Supports RSS-feeds without "name=" tag
* Better daemon handling, only starts one instance
* Init script on Debian (will fix Gentoo soon) is now also able to kill running daemon
_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
fuzzylunkinz
n00b
n00b


Joined: 19 Aug 2011
Posts: 3

PostPosted: Wed Feb 08, 2012 6:42 pm    Post subject: Reply with quote

This looks really cool, I might try it out pretty soon.

Currently on my XBMC media center I use http://flexget.com/ + Deluge, and it works extremely well. Perhaps you hadn't run across it before, and if you had then my apologies :).

I do like your Daemon mode, which is a nice alternative to running a cron job.

EDIT:
Not sure if FlexGet would work on your NAS. If not, even more props to you for this project.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Thu Feb 09, 2012 9:54 am    Post subject: Reply with quote

Hi! I actually hadn't heard of Flexget when I started developing this. I tried googling for some "automatic downloading of torrents from RSS" and stuff, but only found old and abandoned projects that were a little bit bloated for my use. I also wanted to learn more Python (I'm more of a C++-programmer) since I've heard so much praise, so I figured it was perfect to do myself!

The cron solution is definitely simpler to implement, and to some extent more stable/trustworthy, since it doesn't depend on the script not crashing in the long run. I just wanted to have a solution that was "all-in-one", even though it was somewhat of a departure from the Unix way of life (do one thing, and do it well). =)

I'm glad you like it! There is another guy that has forked the project and is doing development on it, so the feature list will probably grow in the future! All great news =)
_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
Gusar
Advocate
Advocate


Joined: 09 Apr 2005
Posts: 2633
Location: Slovenia

PostPosted: Thu Feb 09, 2012 11:13 am    Post subject: Reply with quote

You guys ever heard of rssdler? It's written in python. Provides a daemon mode and all sorts of neat stuff.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Fri Feb 10, 2012 10:26 am    Post subject: Reply with quote

Gusar wrote:
You guys ever heard of rssdler? It's written in python. Provides a daemon mode and all sorts of neat stuff.


Cool, good I hadn't heard of that before I wrote my own ;)
_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Tue Feb 21, 2012 11:19 am    Post subject: New version! Reply with quote

Recent updates:

  • Support for monitoring several sites is now working
  • It is possible to poll these sites at different intervals (specified in minutes)
  • Improved error handling
  • Daemon mode might be a bit dodgy, it hasn't been tested with new support for several sites. Especially regarding shutting the daemon down
  • More options at start (disregard cache, clean cache, dry run (don't download anything), etc)
  • Improved logging


and much more (like code cleanup, etc)! Check it out!

https://github.com/AzP/rsstorrent
_________________
Weeks of coding can save you hours of planning.


Last edited by azp on Sat Oct 13, 2012 9:01 pm; edited 1 time in total
Back to top
View user's profile Send private message
insideyourcouch
n00b
n00b


Joined: 05 Apr 2012
Posts: 1

PostPosted: Thu Apr 05, 2012 1:22 am    Post subject: Reply with quote

Is there anyway to do this through a NAS adapter and an external HDD?
http://www.amazon.co.uk/NAS-adapter-USB-storage-100Mbps/dp/B002U9JKC8/ref=cm_cr_pr_product_top
That?

I know it looks cack, it probably is cack.
I don't have any other luxury than to work with these bare bone cheap accessories.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Thu Apr 05, 2012 9:25 am    Post subject: Reply with quote

insideyourcouch wrote:
Is there anyway to do this through a NAS adapter and an external HDD?
http://www.amazon.co.uk/NAS-adapter-USB-storage-100Mbps/dp/B002U9JKC8/ref=cm_cr_pr_product_top
That?

I know it looks cack, it probably is cack.
I don't have any other luxury than to work with these bare bone cheap accessories.


Well, I guess it's possible, but that all depends on what software/operating system the NAS adapter is running, and how much control you have over it. You need to make sure that you have the rights to install scripts/software on it, and that you have the right to execute it.
_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Sat Oct 13, 2012 9:56 pm    Post subject: Reply with quote

Some improvements have been made, so I've tagged a v0.5 release! I'm currently writing an ebuild for it, so hopefully I can get it into portage eventually... Exciting!
_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
azp
Guru
Guru


Joined: 16 Nov 2003
Posts: 336
Location: Stockholm, Sweden

PostPosted: Thu Oct 25, 2012 2:11 pm    Post subject: Reply with quote

I've now added an ebuild to my own tree, check it out! It's called AzP and can be reached via Layman.

http://git.overlays.gentoo.org/gitweb/?p=user/AzP.git;a=summary
_________________
Weeks of coding can save you hours of planning.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum