View previous topic :: View next topic |
Author |
Message |
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Fri Jan 27, 2012 11:14 am Post subject: Automatically download torrent files using rsstorrent |
|
|
Hi, here's a shameless plug for my own little utility =)
Summary:
I was lacking software that could automatically download torrent files based on a regular expression (or something like it) to my NAS (Excito B3). The torrent download is great, but it kind of defeats the purpose for me, since I still had to log in to my torrentsite, download the torrent, and manually put it on the torrent-folder on the B3.
Therefore I wrote a small program in python that monitors an RSS-feed and downloads all torrent files that match the files you have stated that you would like to download. It works for me on the B3, and you can either start it using screen or the init script that has recently been fixed (both Gentoo and Debian).
The config file is pretty self explanatory, but I have only the software with torrentbytes.net for the time being. I do hope that it works with whatever site you log-in to though.
If you have an account at a site, you have to get the RSS-feed from it (possibly with your hashed password and username in it), and you should then be ready to add it to the rsstorrent config-file and then be ready to roll!
Hope somebody else finds this useful. If you do, please throw me a few comments about what to improve, etc. The things that has been taking most of my development time is actually deciding on how the script should behave, and how the user should be able to control/configure/use it.
If you don't find it useful, please tell me why as well. Perhaps we can find space and time for adding some more features. But my idea is to keep it simple and have it do one thing pretty well.
I created a repository at gitHub, so either you clone it to your NAS/computer (to easily get fixes and changes) using git, or you can just download the files manually from the page.
https://github.com/AzP/rsstorrent
If you want to run it in a screen session (so it keeps on running after you've ended your session):
Code: | 1. Log-in using SSH/start terminal.
2. Run 'screen -S rsstorrent' (use whatever name you want for the session)
3. Start the script (./rsstorrent.py or python rsstorrent.py)
4. Press ctrl+A ctrl+D |
Now the session should be running in the background. Check it is by entering
That should show the name of your session. Reconnect to it the next time you SSH into the box by running
Code: | screen -r rsstorrent |
2011-10-24 Big updates on the code this last week!
* Rewrote config handling to use ConfigParser
* Improved logging (now outputs to ~/.rsstorrent/rsstorrent.log as default or /var/log/rsstorrent/rsstorrent.log if run with init script)
* Fixed crash bugs caused by bad handling of non-resolved connection
Another solution, which would greatly simplify my rsstorrent-script, is to use a cron-job to launch it at specified intervals. That would mean that the code wouldn't have to handle being a daemon and run forever, but at the same time I wanted the code to be "self-contained" after install. Just install, edit your config-file, and then you're ready to go.
2012-01-27 Now with init scripts and proper Daemon mode!
Another "release" has been made as of yesterday. I've updated the script to run in proper Daemon mode, and use pid-files to keep track of itself. I also wrote the init script for the B3, but I forgot to commit it and push it to github, so sadly you can't get a hold of it right now. The Gentoo script is working though, it would be great to see people test it out!
This means that rsstorrent can now be added to automatically start at boot! It's definitely a big improvement in my point of view since I restart my B3/Desktop every now and then, and it's a pain to have to start a screen and run it inside that.
I'd really like some feedback from the Gentoo team regarding the init script, it's the first one I've written and I believe there are improvements to be made. I'm not entirely sure about the way I'm handling running it as a daemon.
Features planned ahead:
- Re-read (while running) the config file after it has been changed and supply log output regarding the new search patterns.
_________________ Weeks of coding can save you hours of planning. |
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Tue Feb 07, 2012 3:25 pm Post subject: |
|
|
Updated version uploaded!
* Supports RSS-feeds without "name=" tag
* Better daemon handling, only starts one instance
* Init script on Debian (will fix Gentoo soon) is now also able to kill running daemon _________________ Weeks of coding can save you hours of planning. |
|
Back to top |
|
|
fuzzylunkinz n00b
Joined: 19 Aug 2011 Posts: 3
|
Posted: Wed Feb 08, 2012 6:42 pm Post subject: |
|
|
This looks really cool, I might try it out pretty soon.
Currently on my XBMC media center I use http://flexget.com/ + Deluge, and it works extremely well. Perhaps you hadn't run across it before, and if you had then my apologies .
I do like your Daemon mode, which is a nice alternative to running a cron job.
EDIT:
Not sure if FlexGet would work on your NAS. If not, even more props to you for this project. |
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Thu Feb 09, 2012 9:54 am Post subject: |
|
|
Hi! I actually hadn't heard of Flexget when I started developing this. I tried googling for some "automatic downloading of torrents from RSS" and stuff, but only found old and abandoned projects that were a little bit bloated for my use. I also wanted to learn more Python (I'm more of a C++-programmer) since I've heard so much praise, so I figured it was perfect to do myself!
The cron solution is definitely simpler to implement, and to some extent more stable/trustworthy, since it doesn't depend on the script not crashing in the long run. I just wanted to have a solution that was "all-in-one", even though it was somewhat of a departure from the Unix way of life (do one thing, and do it well). =)
I'm glad you like it! There is another guy that has forked the project and is doing development on it, so the feature list will probably grow in the future! All great news =) _________________ Weeks of coding can save you hours of planning. |
|
Back to top |
|
|
Gusar Advocate
Joined: 09 Apr 2005 Posts: 2665 Location: Slovenia
|
Posted: Thu Feb 09, 2012 11:13 am Post subject: |
|
|
You guys ever heard of rssdler? It's written in python. Provides a daemon mode and all sorts of neat stuff. |
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Fri Feb 10, 2012 10:26 am Post subject: |
|
|
Gusar wrote: | You guys ever heard of rssdler? It's written in python. Provides a daemon mode and all sorts of neat stuff. |
Cool, good I hadn't heard of that before I wrote my own _________________ Weeks of coding can save you hours of planning. |
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Tue Feb 21, 2012 11:19 am Post subject: New version! |
|
|
Recent updates:
- Support for monitoring several sites is now working
- It is possible to poll these sites at different intervals (specified in minutes)
- Improved error handling
- Daemon mode might be a bit dodgy, it hasn't been tested with new support for several sites. Especially regarding shutting the daemon down
- More options at start (disregard cache, clean cache, dry run (don't download anything), etc)
- Improved logging
and much more (like code cleanup, etc)! Check it out!
https://github.com/AzP/rsstorrent _________________ Weeks of coding can save you hours of planning.
Last edited by azp on Sat Oct 13, 2012 9:01 pm; edited 1 time in total |
|
Back to top |
|
|
insideyourcouch n00b
Joined: 05 Apr 2012 Posts: 1
|
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Thu Apr 05, 2012 9:25 am Post subject: |
|
|
Well, I guess it's possible, but that all depends on what software/operating system the NAS adapter is running, and how much control you have over it. You need to make sure that you have the rights to install scripts/software on it, and that you have the right to execute it. _________________ Weeks of coding can save you hours of planning. |
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
Posted: Sat Oct 13, 2012 9:56 pm Post subject: |
|
|
Some improvements have been made, so I've tagged a v0.5 release! I'm currently writing an ebuild for it, so hopefully I can get it into portage eventually... Exciting! _________________ Weeks of coding can save you hours of planning. |
|
Back to top |
|
|
azp Guru
Joined: 16 Nov 2003 Posts: 456 Location: Sweden
|
|
Back to top |
|
|
|