Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Download ISOs faster and error free with metalink
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
twanj
n00b
n00b


Joined: 08 Jul 2007
Posts: 6

PostPosted: Sun Jul 08, 2007 12:16 am    Post subject: HOWTO: Download ISOs faster and error free with metalink Reply with quote

metalinks are an XML format listing mirrors, p2p locations, checksums, & other stuff in one file used by download managers.

downloads are usually quite fast, and if aria2 is used, they will be error free and repairable.

1. Install aria2
2. http://www.metalinker.org/samples.html#isos has a large number of ISOs, including Gentoo
3. Download the Metalink for the ISO you want with 'aria2c URLtoMetalink'

if you download the .metalink file, you can use it locally with 'aria2c -M metalink'

there's a patch for Bouncer if gentoo wanted to use this for ISO downloads. different sources can be given priority, and downloads can be filtered by location and other things.

here's an example of what one looks like, minus the chunk checksums used for repairs.

Code:
<?xml version="1.0" encoding="utf-8"?>
<metalink version="3.0" generator="Metalink Editor version 0.4.1" xmlns="http://www.metalinker.org/">
  <publisher>
    <name>Gentoo</name>
    <url>http://www.gentoo.org/</url>
  </publisher>
  <version>2007.0</version>
  <files>
    <file name="livecd-i686-installer-2007.0.iso">
      <size>734308352</size>
      <os>Linux-x86</os>
      <verification>
        <hash type="md5">7a21c2f09edd51612fb2cd7984dd3577</hash>
        <hash type="sha1">d876060d2b322b9afffdcd81da2ec8433e489d93</hash>
      </verification>
      <resources>
        <url type="http" location="us" preference="100">http://distro.ibiblio.org/pub/linux/distributions/gentoo/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="kr" preference="100">http://ftp.gentoo.or.kr/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="tw" preference="100">http://ftp.isu.edu.tw/pub/Linux/Gentoo/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="ro" preference="100">http://ftp.roedu.net/pub/mirrors/gentoo.org/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="uk" preference="100">http://gentoo.blueyonder.co.uk/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="jp" preference="100">http://gentoo.gg3.net/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="ca" preference="100">http://gentoo.mirrored.ca/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="us" preference="100">http://gentoo.mirrors.pair.com/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" preference="100">http://gentoo.scphost.com/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="de" preference="100">http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="jp" preference="100">http://mirror.gentoo.gr.jp/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="it" preference="100">http://www.die.unipd.it/pub/Linux/distributions/gentoo-sources/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="us" preference="100">http://www.gtlib.gatech.edu/pub/gentoo/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="br" preference="100">http://www.las.ic.unicamp.br/pub/gentoo/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
        <url type="http" location="uk" preference="100">http://www.mirrorservice.org/sites/www.ibiblio.org/gentoo/releases/x86/2007.0/livecd/livecd-i686-installer-2007.0.iso</url>
      </resources>
    </file>
  </files>
</metalink>
Back to top
View user's profile Send private message
twanj
n00b
n00b


Joined: 08 Jul 2007
Posts: 6

PostPosted: Mon Jul 09, 2007 11:27 pm    Post subject: Reply with quote

DownThemAll! 1.0 Beta now supports metalink downloads from mirrors and full file checksum verification.
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2159
Location: The Peanut Gallery

PostPosted: Tue Jul 10, 2007 11:12 am    Post subject: Reply with quote

Looks interesting. There's no DTD mentioned in your sample file. That means it's not valid XML aiui. Do you believe this file-format is necessary, or could you see it being done with say an ini-style format? (or anythng but XML ;)
Back to top
View user's profile Send private message
twanj
n00b
n00b


Joined: 08 Jul 2007
Posts: 6

PostPosted: Tue Jul 10, 2007 2:20 pm    Post subject: Reply with quote

A schema is at http://www.metalinker.org/schema/3.0/metalink.xsd & spec/other info at http://www.metalinker.org/

What do you have against XML? :)
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2159
Location: The Peanut Gallery

PostPosted: Sun Jul 15, 2007 5:15 pm    Post subject: Reply with quote

twanj wrote:
A schema is at http://www.metalinker.org/schema/3.0/metalink.xsd & spec/other info at http://www.metalinker.org/


Quote:
What do you have against XML? :)

http://modeemi.fi/~tuomov/b////archives/2006/04/14/T18_33_42/
http://modeemi.fi/~tuomov/b///archives/2007/01/20/T11_58_29/
..pretty much explain it. I like Gentoo GuideXML for instance as it is simple and clean, but usually XML files are way too verbose/ content-light for my liking. I blame MS for making it so bloated in their usage of it; it's ended up being something difficult for people to edit.

Most of all i like rst (as a format) and TeX for complex stuff. For configuration I prefer simple plaintext, although I haven't yet seen the one true format (can't find the url for that one but i don't really like it either) ;)
Back to top
View user's profile Send private message
twanj
n00b
n00b


Joined: 08 Jul 2007
Posts: 6

PostPosted: Fri Jul 20, 2007 12:19 am    Post subject: Reply with quote

Metalink is aimed at browsers and other programs that already support XML. XML seems to fit fine with what it's doing, it's been working fine for years, so I doubt it will change soon unless there were huge benefits for doing so. :wink:
Back to top
View user's profile Send private message
twanj
n00b
n00b


Joined: 08 Jul 2007
Posts: 6

PostPosted: Wed Jan 16, 2008 10:07 pm    Post subject: Reply with quote

About 20 apps now support downloading w/ metalinks now, including KGet in KDE 4.0, the DTA Firefox extension, aria2 (CLI), wxDownload Fast, GetRight, Net Transport, FDM, Orbit, Retriever, and Speed Download.

There are a bunch of tools for automating the creation of metalinks too.
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5610
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Thu Jan 24, 2008 5:06 pm    Post subject: Reply with quote

*subscribes*

thanks !
I need to try this the next time I download a livecd
_________________
Unofficial minimal livecd x86/amd64 w/reiser4+truecrypt (by Neo2)
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum