View previous topic :: View next topic |
Author |
Message |
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
Posted: Mon Feb 03, 2003 12:33 am Post subject: Bit torrent |
|
|
wesent sure where to put this, so i put it in here. there was talk about the P2P File sharing on the forums someware. Im not talking about music and movies, but apps. gentoo sources. bittorrent may be the answer. if the developers could set it up so that a .torrent file is downloaded by emerge and then tell bittorrent to open it, save the file to the distfiles dir, and then do what has to be done, would that not speed up things? even if lets say 50 users ware to give 4kbytes a seccond of bandwidth up (maybe there is no limit, emerging now and ibiblio is slow! ) thats 200kbytes. 1.6mb/s of bandwidth. now if you ware to add lets say an other 50 users, and give them 16kbytes (128kbits) upload bandwidth, you have 12.5mb/s of bandwidth! you use ibilio and others for the mirroring of the original files, their then hashed to make sure there correct. keep the original rsync mirror, so no messing with files there. Hmmm. any questions? any ideas? any problems in getting this to work? is bit torrent command line compatable? _________________ Lotas T Smartman
www.lotas-smartman.net
www.the-hairy-one.com
www.lsn-blog.tk
Dual Athlon 2Gz, 1Gb ram, 120Gb hdd, GeForce FX5200, DVD+R/-R/+RW/-RW, CDR/RW |
|
Back to top |
|
|
mawst Tux's lil' helper
Joined: 23 Nov 2002 Posts: 101 Location: MN, USA
|
Posted: Mon Feb 03, 2003 1:01 am Post subject: Torrents |
|
|
While I appluad your idea, I don't find it at all realistic.
There are far too many "what if's" involved for this to ever become a reality.
Furthermore, the current mirroring system for Gentoo and the rsync system, although still in it's infancy do a pretty damn good job.
You have to realize that major mirroring servers such as ibiblio serve MASSIVE amounts of bandwidth and I doubt that people using some sort of torrent system could match the bandwidth they provide. |
|
Back to top |
|
|
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
Posted: Mon Feb 03, 2003 1:08 am Post subject: |
|
|
well although i agree that the system we have is good, it does have its flaws. if ibiblio is attacked, by lets say a ddos attack, what then? The nearest mirror to me is in the uk and they only have rc1. there are a good amount of distfiles, but not all of them are there. I know that ibilio does serve quite a lot of bandwidth, but it is also quite slow now. could sourceforge do some hosting? they have a few mirror sites world wide now. if you used all three (including the torrants idea) you could increese max bandwidth. i know its not for everyone, but it may work. im still building the system as we speek now, but, i have seen this online and working. there is quite a good reason for it to successed. i see peopel sharing 600mb videos over 128k up connections, and because there has been so many people downloading this file, he is not being hit that much. his server is coping. i want to try it on some stuff, but most of mine is under 3 - 5 mb. ahhh, its was just a though! if it worked, i would have helped a community that helped me! _________________ Lotas T Smartman
www.lotas-smartman.net
www.the-hairy-one.com
www.lsn-blog.tk
Dual Athlon 2Gz, 1Gb ram, 120Gb hdd, GeForce FX5200, DVD+R/-R/+RW/-RW, CDR/RW |
|
Back to top |
|
|
Oo.et.oO Apprentice
Joined: 23 Apr 2002 Posts: 291 Location: burlington
|
Posted: Tue Feb 04, 2003 1:34 am Post subject: Re: Torrents |
|
|
mawst wrote: | While I appluad your idea, I don't find it at all realistic.
There are far too many "what if's" involved for this to ever become a reality.
|
enlighten us. what are the first five (for instance) what ifs that you are anticipating?
mawst wrote: |
You have to realize that major mirroring servers such as ibiblio serve MASSIVE amounts of bandwidth and I doubt that people using some sort of torrent system could match the bandwidth they provide. |
i believe you are mistaken here.
the more people that download a torrent stream the more the upload capacity overall for other uses.
go here for info:
http://bitconjurer.org/BitTorrent/
i think this is a great idea. emerge could use this as an option (since it may be hard to get bittorrent going for the bootstrap and base system portions (one what-if)).
i am not sure of how to open a torrent stream via command line. but i'm sure that this tool will be trivial to write. adding this functionality to emerge might be cool.
great idea.
Last edited by Oo.et.oO on Wed Feb 05, 2003 9:04 pm; edited 1 time in total |
|
Back to top |
|
|
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
Posted: Tue Feb 04, 2003 2:35 am Post subject: |
|
|
i was actually playing with bit torrent, and its fairly easy to get a torrent started from command like. there are 2 options. either a .torrent file you have downloaded (or lets say have been emailed, clicked on though mozilla, etc) or a direct link to the .torrent file on the server. the direct link may be the best way, as it meens lest downloading of .torrent files. Also easy update with that. secondly, bit torrent does need a few things. it has the following depends: Code: | DEPEND=">=dev-python/wxPython-2.2
>=dev-lang/python-2.1"
| and what ever they depend on too. There was about 12mb of downloads done on my end. it would meen a slight reform as far as i can see. insted of listing a tar.gz or tar.bz file for each item to be downloaded, a .torrent file is placed on the server. then, you can just use bittorrent to download that and run it. You can change between wget and other downloaders fairly easly, so bit torrent could be thrown in there, but stuff would have to be passed to it. im guessing the following: Code: | MD5 2bb60b7594a416f2c593923ce446160b linux-2.4.19.tar.bz2 26042494
MD5 59ff0f443bed8cadb01addda59f51922 patches-2.4.19-gentoo-r10.tar.bz2 2665464 | insted of listing a tar.bz in there, you would have the torrent file. then thats passed to the client, that downloades the file, and then md5 kicks in and checks. would md5 have to check at all? hmmm. any other ideas? _________________ Lotas T Smartman
www.lotas-smartman.net
www.the-hairy-one.com
www.lsn-blog.tk
Dual Athlon 2Gz, 1Gb ram, 120Gb hdd, GeForce FX5200, DVD+R/-R/+RW/-RW, CDR/RW |
|
Back to top |
|
|
masseya Bodhisattva
Joined: 17 Apr 2002 Posts: 2602 Location: Baltimore, MD
|
Posted: Tue Feb 04, 2003 7:00 am Post subject: |
|
|
Whoa, Oo.et.oO, please remain civil. There's no reason to use that kind of tone on these forums. If we are able to say things in a kinder manner, these forums will be a great resource for information. _________________ if i never try anything, i never learn anything..
if i never take a risk, i stay where i am.. |
|
Back to top |
|
|
masseya Bodhisattva
Joined: 17 Apr 2002 Posts: 2602 Location: Baltimore, MD
|
Posted: Wed Feb 05, 2003 9:14 pm Post subject: |
|
|
Oo.et.oO, thanks for changing some of the language there. _________________ if i never try anything, i never learn anything..
if i never take a risk, i stay where i am.. |
|
Back to top |
|
|
mawst Tux's lil' helper
Joined: 23 Nov 2002 Posts: 101 Location: MN, USA
|
Posted: Thu Feb 06, 2003 2:48 pm Post subject: ... |
|
|
Oo.et.oO
I'm fully aware of the nature of bit torrent.
What I'm saying is, as Gentoo continues to mature, I doubt that a torrent system would be able to put out the same upload bandwidth as several large mirroring sites (like ibiblio).
In other words, I'm sure we will continue to see more and more FAST mirrors emerge to support the Gentoo project, and things will be just fine.
The idea itself of using a torrent system is messy in the first place imo. And I for one do not want to use my limited bandwidth to send other people files that they could have gotten from someplace like ibiblio. |
|
Back to top |
|
|
mawst Tux's lil' helper
Joined: 23 Nov 2002 Posts: 101 Location: MN, USA
|
Posted: Thu Feb 06, 2003 2:50 pm Post subject: ... |
|
|
lotas wrote: | well although i agree that the system we have is good, it does have its flaws. if ibiblio is attacked, by lets say a ddos attack, what then? The nearest mirror to me is in the uk and they only have rc1. there are a good amount of distfiles, but not all of them are there. I know that ibilio does serve quite a lot of bandwidth, but it is also quite slow now. could sourceforge do some hosting? they have a few mirror sites world wide now. if you used all three (including the torrants idea) you could increese max bandwidth. i know its not for everyone, but it may work. im still building the system as we speek now, but, i have seen this online and working. there is quite a good reason for it to successed. i see peopel sharing 600mb videos over 128k up connections, and because there has been so many people downloading this file, he is not being hit that much. his server is coping. i want to try it on some stuff, but most of mine is under 3 - 5 mb. ahhh, its was just a though! if it worked, i would have helped a community that helped me! |
If the system could be implemented as an alternative, I would be for it, simply to aleviate some of the strain from the normal mirrors.
But completely switching to me is out of the realm of being acceptable. |
|
Back to top |
|
|
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
Posted: Thu Feb 06, 2003 7:17 pm Post subject: |
|
|
i know that it is not posbile for torrent to take over fully. if there was a system that you downloaded off a mirror site, and lets say an other few users, like torrents, then that would be great. you have lets say 25 - 50% of your stuff comming from either one or several mirrors, and some from users (close to you). Like ibiblio is in the US. If, lets say, i want to download gnome2.2 (when ebuilds premit) i could download some from ibiblio, some from other local mirrors (mirror.ac.uk) and some from users from my ISP (NTL) who are on cable and have the files. In the us, you have more users. AOL could have thousands of users using gentoo. its not a quick answer, but if that could work, it could be an answer. it could be like getright on windows. up to 10 mirrors, but some could be P2P users. it would work out who was quicker due to ping responces, and number of hops. _________________ Lotas T Smartman
www.lotas-smartman.net
www.the-hairy-one.com
www.lsn-blog.tk
Dual Athlon 2Gz, 1Gb ram, 120Gb hdd, GeForce FX5200, DVD+R/-R/+RW/-RW, CDR/RW |
|
Back to top |
|
|
mawst Tux's lil' helper
Joined: 23 Nov 2002 Posts: 101 Location: MN, USA
|
Posted: Fri Feb 07, 2003 2:06 pm Post subject: ... |
|
|
Code it!. |
|
Back to top |
|
|
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
Posted: Fri Feb 07, 2003 2:13 pm Post subject: |
|
|
well, if i could i would. i know a little C++, but mainly program in VB because of college. _________________ Lotas T Smartman
www.lotas-smartman.net
www.the-hairy-one.com
www.lsn-blog.tk
Dual Athlon 2Gz, 1Gb ram, 120Gb hdd, GeForce FX5200, DVD+R/-R/+RW/-RW, CDR/RW |
|
Back to top |
|
|
Oo.et.oO Apprentice
Joined: 23 Apr 2002 Posts: 291 Location: burlington
|
Posted: Fri Feb 07, 2003 4:55 pm Post subject: Re: ... |
|
|
mawst wrote: |
If the system could be implemented as an alternative, I would be for it, simply to aleviate some of the strain from the normal mirrors.
But completely switching to me is out of the realm of being acceptable. |
agreed! i also concede to your previous response. i see your point(s). |
|
Back to top |
|
|
Caffeine Guru
Joined: 17 Jul 2002 Posts: 401 Location: Melbourne, Australia
|
Posted: Mon Mar 03, 2003 2:06 pm Post subject: |
|
|
Sounds like an interesting idea. Would it be easiest to start with portage snapshots? Easier to torrent one file, rather than the portage directory I'd imagaine? |
|
Back to top |
|
|
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
Posted: Mon Mar 03, 2003 2:56 pm Post subject: |
|
|
I was looking for a copy of smootwall off their servers, and i found they they are offering their ISO's on the Edonkey 2000 network. Edonkey works on windows and linux, and a mac i think, so maybe sharing the ISO files only (since there about 400mb each) over edonkey or something like that could help. edonkey is pretty cool in that you have to share stuff. you cant just leach off someone. i found a way to leach off bit torrent. but with edonkey you kinda have to share or download at really crap speeds! _________________ Lotas T Smartman
www.lotas-smartman.net
www.the-hairy-one.com
www.lsn-blog.tk
Dual Athlon 2Gz, 1Gb ram, 120Gb hdd, GeForce FX5200, DVD+R/-R/+RW/-RW, CDR/RW |
|
Back to top |
|
|
dreamer3 Guru
Joined: 24 Sep 2002 Posts: 553
|
Posted: Mon Mar 03, 2003 3:45 pm Post subject: |
|
|
lotas wrote: | AOL could have thousands of users using gentoo. |
AOL user = Gentoo user
Doesn't fit the image in my head very well. |
|
Back to top |
|
|
lotas Tux's lil' helper
Joined: 23 Jul 2002 Posts: 121 Location: Tallaght, Dublin, Ireland
|
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|