| View previous topic :: View next topic |
| Author |
Message |
salimfadhley
n00b
Joined: 23 Mar 2003
Posts: 44
Location: London, England
|
 Posted: Tue Jun 03, 2003 7:59 pm Post subject: Why not distribute those LiveCDs with BitTorrent |
 |
|
I'm downloading the PPC liveCD via a BitTorrent set up by some guy on Slashdot - it's not working too well because he is probably running his tracker off an ADSL connection - even BT can suffer the slashdot effect! Even so, I am getting better performance from the BT download than I did from the Gentoo mirrors.
It occured to me that if Gentoo has 6 willing big-bandwidth mirrors, this bandwidth could be far more effectivly used by BitTorrent than simple FTP or HTTP downloads. The main benefit to Gentoo is that this would substantially lower the cost of serving each LiveCD.
Of course not everybody can run BitTorrent, but the regular FTP or HTTP downloads could be opened up a few days after the rush has died down.
For those who have not yet tried it, Gentoo comes with an ebuild for bitTorrent - just emerge it and enjoy! |
|
| Back to top |
|
 |
magnet
Guru
Joined: 16 Mar 2003
Posts: 582
Location: france
|
| Posted: Wed Jun 04, 2003 1:44 pm Post subject: |
|
|
let's try it.
_________________
every step aim at glory. |
|
| Back to top |
|
|
iceburglar
n00b
Joined: 05 Feb 2003
Posts: 10
Location: Boston, MA (God's Country)
|
| Posted: Fri Jun 13, 2003 4:51 am Post subject: If it's good enough for mandrake.... |
|
|
I've been searching for posts on Bittorrent, and it looks like the idea of using it with portage has been shot down, with 2 issues fingered as problems: 1.) bittorrent not being designed for small files, and 2.) bittorrent not being designed for dial-up users.
https://forums.gentoo.org/viewtopic.php?t=58955
First off, I think the LiveCD's and the stage tarballs would be perfect candidates for Bittorrent. I should have my webserver back up in about a week or two, and will gladly offer up space for files and counters.
Back to using with portage, I'm not to familiar enough with bittorrent to address issue 1, but I would think it could be worked around (tar up groups of related files, maybe?). Issue 2, however seems strange. I think the same argument could be made about Gentoo not being designed for dial-up users. Perhaps implementing it as an option, and using it as a transparent layer (wget accesses a local bittorrent servlet, which in turn goes handles the downloading and returns the files to wget).
Personally, the major issue I would foresee would be in keeping files up to date among all the various peers serving them up, but I think bittorrent has some verification functions built in.
This is all just conjecture, and purely MHO, but I think bittorrent, in conjunction with backgrounded emerge downloads would be a killer app for Gentoo.
_________________
--
Iceburglar;
To the world, you may just be one person,
But to one person, you may just be the world. - Source Unknown. |
|
| Back to top |
|
|
progster
Apprentice
Joined: 16 Jul 2002
Posts: 271
|
| Posted: Fri Jun 13, 2003 7:17 am Post subject: |
|
|
bt isn't fast for people with a small upload (like when you're on a cable modem in belgium  ), no fast upload, no fast download... so that's not really fair now is it 
edit and local mirrors are FAST, I got my gentoo cd from belnet (belgium mirror) at 800kbs....
~Progster |
|
| Back to top |
|
|
iceburglar
n00b
Joined: 05 Feb 2003
Posts: 10
Location: Boston, MA (God's Country)
|
| Posted: Sun Jun 15, 2003 10:54 am Post subject: BitTorrent is a no-brainer for LiveCD's and Stage Tarballs |
|
|
| progster wrote: | | bt isn't fast for people with a small upload (like when you're on a cable modem in belgium ), no fast upload, no fast download... so that's not really fair now is it |
Pretty much every cable provider throttles their users' upstream bandwidth pretty heavily (like 1.5M/128k). BT will break downloads up into multiple parts, each of which can be served up from different hosts. So, 128kbit may be pretty slow, but 128kbit times 6 can be very fast, AND more importantly, the load will be pushed out onto user connections, rather than crushing the Gentoo servers and mirrors.
In any case, I think using bt for Portage/e-builds would have to be optional and abstracted to be transparent to the main emerge/ebuild/portage functionality. Honestly, I don't think the benefits of this would outweigh the costs anyway.
However, using BitTorrent for the LiveCD's and Stage Tarballs is a no-brainer, and I am very surprised Gentoo is not already doing it. I downloaded Mandrake ISO's this way, and it worked great, allowing full saturation of my 1.5M download bandwidth for just about the entire time. Also, not a single byte of the download came from a Mandrake server!
If you think about it, it is a win-win situation:
-- Wins for Gentoo
1. The largest files are decentralized and served up by multitudes of user machines rather than crushing one centralized server.
2. The bandwidth that is used is slack capacity that users have already paid for, but aren't using, rather than throughput Gentoo has to pay for.
3. Great Scaleability: More users requesting DL's = more users available to serve UL's
-- Wins for users:
1. Less reliance on centralized servers and mirrors.
2. Ability to multiplex several connections to maximize throughput from various sources.
3. Breaking up larger files into smaller chunks means corrupt chunks can be recovered rather than having to start over if the monolithic file is corrupted in transit.
4. You'll be giving back to the Open Source Community, and in a big way.
I'm sure there are a lot more selling points for both camps.
Anyway, I'd like to propose a pilot test: For the next full release, for the first few days, offer some of the LiveCD's exclusively via BitTorrent. I think good pilot test candidates will be the architecture specific versions(athlon, athlon-mp, i586, etc.) of the LiveCD's and stage 2/3 tarballs. By offering them via BT first, you'll provide an incentive for users to try it (everybody wants to be first ;p), yet still have the familiar DL method a few days later.
Continue serving up the gentoo-basic LiveCD's and stage 1 tarballs since they are smaller.
Alternatively, you could offer both methods, and compare them.
_________________
--
Iceburglar;
To the world, you may just be one person,
But to one person, you may just be the world. - Source Unknown. |
|
| Back to top |
|
|
wHAcKer
Apprentice
Joined: 18 Oct 2002
Posts: 228
Location: Grimbergen, Belgium
|
| Posted: Sun Jun 15, 2003 11:17 am Post subject: |
|
|
It's quite some idea, but there always are users that CAN NOT spare even a few megabytes upload. I'm thinking about users who have the permission to download their livecd's/source files at their office (all the upload is needed to handle incoming requests, even when there's loads of bandwith) and users that have a limitation in upload (e.g. my provider limits uploads to 1,5 gig / 30 days, if you go over that you'll be on smallband untill you're under the 1.5 gig again, smallband is MAX 3k download 3k upload but they also make sure you can't game properly...)
I think since gentoo is becoming popular, more and more mirrors will rise and this will reduce the problem greatly. (I've seen a few mirrors rise in my neighbourhood e.g. skynet (Belgium)) |
|
| Back to top |
|
|
jrz
Apprentice
Joined: 19 Mar 2003
Posts: 272
Location: Sacramento, CA USA
|
| Posted: Sun Jun 15, 2003 11:52 am Post subject: |
|
|
you can limit the upload with "--max_upload_rate n" don't tell anyone  i suggest setting it -10 of what your max is so it doesn't totally ruin the inter-web for you.
_________________
Press F1 for Help. |
|
| Back to top |
|
|
puggy
Bodhisattva
Joined: 28 Feb 2003
Posts: 1992
Location: Oxford, UK
|
| Posted: Sun Jun 15, 2003 12:04 pm Post subject: |
|
|
I get stuff from mirror.ac.uk
lightning fast. Especally when I connect from uni and I get all SuperJANETs bandwidth. 600meg in 3 minutes and speed still increasing. 
And I couldn't use my uni connection to use bittorrent so this would be crap for me. 
Puggy
_________________
Where there's open source , there's a way. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
