Automate mirror usage and checksums for ISO downloads

Discuss all artwork and development - Suggestions needed

Moderator: Moderators

Automate mirror usage and checksums for ISO downloads

Postby twanj » Sun Apr 22, 2007 19:43

Sabayon could use metalink for ISO downloads. It's an XML file format listing mirrors and checksums for higher availability and integrity. It corrects errors during ftp/http downloads. A few older ISO are listed at http://www.metalinker.org/samples.html

There are automated tools for creating metalinks. More info at http://en.wikipedia.org/wiki/Metalink
twanj
Baby Hen
 
Posts: 8
Joined: Sun Apr 22, 2007 19:35

Postby twanj » Tue Jun 12, 2007 6:13

Updated metalinks for 3.3 ISOs are at http://www.metalinker.org/samples.html#sabayon

These list all mirrors & checksums (chunk checksums are omitted here to save space, but are in the metalinks to repair downloads. About 20 programs support metalink, the best is probably aria2 but it's command line. openSUSE, OpenOffice.org, cURL and many other distributions use metalink.

Using metalinks with checksums means you don't have errors, and you don't have to instruct users to do things like this: http://www.sabayonlinux.org/wiki/index. ... our_MD5SUM

Here's what the metalink files look like:

Code: Select all
<?xml version="1.0" encoding="utf-8"?>
<metalink version="3.0" generator="Metalink Editor version 1.0.0" xmlns="http://www.metalinker.org/">
  <publisher>
    <name>SabayonLinux</name>
    <url>http://www.sabayonlinux.org/</url>
  </publisher>
  <identity>SabayonLinux</identity>
  <version>3.3</version>
  <description>SabayonLinux is a Gentoo based Linux distro.</description>
  <files>
    <file name="SabayonLinux-x86-3.3.iso">
      <size>3349598208</size>
      <os>Linux-x86</os>
      <verification>
        <hash type="md5">e8f2a1100f2292edec55f3296c1d1f5b</hash>
        <hash type="sha1">911d745d96517e59cc5ce0cb52088d1b4bbab3c1</hash>
      </verification>
      <resources>
        <url type="http" location="au" preference="100">http://mirror.internode.on.net/pub/sabayonlinux/SabayonLinux-x86-3.3.iso</url>
        <url type="ftp" location="au" preference="100">ftp://mirror.internode.on.net/pub/sabayonlinux/SabayonLinux-x86-3.3.iso</url>
        <url type="http" location="it" preference="100">http://bo.mirror.garr.it/mirrors/sabayonlinux/SabayonLinux-x86-3.3.iso</url>
        <url type="ftp" location="it" preference="100">ftp://bo.mirror.garr.it/mirrors/sabayonlinux/SabayonLinux-x86-3.3.iso</url>
        <url type="ftp" location="us" preference="100">ftp://dotdot.mine.nu/sabayon/SabayonLinux-x86-3.3.iso</url>
        <url type="https" location="dk" preference="100">https://binstar.org/sabayon/SabayonLinux-x86-3.3.iso</url>
        <url type="http" location="de" preference="100">http://sabayon.domaindorf.de/SabayonLinux-x86-3.3.iso</url>
        <url type="http" location="us" preference="100">http://mirror.cs.vt.edu/pub/SabayonLinux/SabayonLinux-x86-3.3.iso</url>
        <url type="ftp" location="us" preference="100">ftp://mirror.cs.vt.edu/pub/SabayonLinux/SabayonLinux-x86-3.3.iso</url>
        <url type="ftp" location="pt" preference="100">ftp://glua.ua.pt/sabayon/SabayonLinux-x86-3.3.iso</url>
      </resources>
    </file>
  </files>
</metalink>
twanj
Baby Hen
 
Posts: 8
Joined: Sun Apr 22, 2007 19:35

Postby voxiac » Tue Jun 12, 2007 14:15

How do you include bittorrent mirrors?
Is it like:
Code: Select all
<url type="bittorrent" preference="100">http://linuxtracker.org/download.php?id=3744&name=SabayonLinux-x86-3.3.iso.torrent</url>

Will this work?

Also here will come the problem of instructing users about right download managers...
Don't get me wrong though - I like the idea.
voxiac
Advanced Hen
 
Posts: 218
Joined: Sat Feb 10, 2007 17:05
Location: Denmark

Postby twanj » Wed Jun 13, 2007 2:20

voxiac wrote:How do you include bittorrent mirrors?
Is it like:
Code: Select all
<url type="bittorrent" preference="100">http://linuxtracker.org/download.php?id=3744&name=SabayonLinux-x86-3.3.iso.torrent</url>

Will this work?

Also here will come the problem of instructing users about right download managers...
Don't get me wrong though - I like the idea.


Yes, that will work.

Instruction is a problem, since it is less familiar than bittorrent. But a lot of people are familiar with neither. Metalink also has the possibility of being supported by a lot of programs that don't support torrents...Right now, only about 20 programs support it, but there are at least some on all platforms.

Here's how other distros instruct their users, see above for how they also explain bittorrent.

http://en.opensuse.org/Released_Version#Metalink
twanj
Baby Hen
 
Posts: 8
Joined: Sun Apr 22, 2007 19:35

Postby twanj » Thu Jun 14, 2007 5:33

metalinks for 3.4loop2 are at http://www.metalinker.org/samples.html#sabayon now.
twanj
Baby Hen
 
Posts: 8
Joined: Sun Apr 22, 2007 19:35

Re: Automate mirror usage and checksums for ISO downloads

Postby teedog » Sun Sep 09, 2007 2:31

Here's a metalink for SabayonLinux-x86-3.4f.iso.
http://www.metalinker.org/samples.html#sabayon
Code: Select all
<?xml version="1.0" encoding="utf-8"?>
<metalink version="3.0" generator="Metalink Generator v1.00.0037" xmlns="http://www.metalinker.org/">
  <publisher>
    <name>Sabayon Linux</name>
    <url>http://www.sabayonlinux.org/</url>
  </publisher>
  <files>
    <file name="SabayonLinux-x86-3.4f.iso">
      <os>Linux-x86</os>
      <verification>
        <hash type="md5">9fabc6ce4a033f55b537ed19cf86be8e</hash>
      </verification>
      <resources>
        <url type="ftp" preference="100">ftp://glua.ua.pt/sabayon/SabayonLinux-x86-3.4f.iso</url>
        <url type="ftp" preference="100">ftp://mirror.aarnet.edu.au/pub/SabayonLinux/SabayonLinux-x86-3.4f.iso</url>
        <url type="http" preference="100">http://bo.mirror.garr.it/mirrors/sabayonlinux/SabayonLinux-x86-3.4f.iso</url>
        <url type="http" preference="100">http://cesium.di.uminho.pt/pub/sabayon/SabayonLinux-x86-3.4f.iso</url>
        <url type="http" preference="100">http://cross-lfs.sabayonlinux.org/SabayonLinux-x86-3.4f.iso</url>
        <url type="http" preference="100">http://mirror.internode.on.net/pub/sabayonlinux/SabayonLinux-x86-3.4f.iso</url>
        <url type="bittorrent" preference="100">http://www.linuxtracker.org/download.php?id=4564&name=SabayonLinux-x86-3.4f.iso.torrent</url>
      </resources>
    </file>
  </files>
</metalink>
teedog
Baby Hen
 
Posts: 1
Joined: Sun Sep 09, 2007 2:29

Re: Automate mirror usage and checksums for ISO downloads

Postby Nicolas » Fri Sep 28, 2007 22:52

I have metalinks of the newer 3.4 miniEditions at http://impfarm.imp.org/~nico/sabayon/ (the "http-only" metalink was to test a download manager bug).

I'll see if I can add chunk checksums later.
Nicolas
Baby Hen
 
Posts: 15
Joined: Sun Apr 15, 2007 23:09

Re: Automate mirror usage and checksums for ISO downloads

Postby WarraWarra » Sun Nov 04, 2007 8:21

If anyone wants to or have the time to make new metalinks or update the current ones for mostly new versions of SL or all versions just shout as the ibiblio.org server now has a metalink folder so we can post them on there , redistribute them from there etc etc . Just buzz me / send me the links and I will download / post them there or something like this.

Just did not have time myself to update the metlinks with all server links yet.

PS> When making / updating the metalinks.
:oops: Just ignore the ibiblio.org http link for downloads as it still has internal linking issues or something and working on it, note a few missing iso files in http side.
:idea: Should actually for the time being temporarily remove the http ibiblio.org from the SL downloads page until fixed.

http://distro.ibiblio.org/pub/linux/dis ... Metalinks/
ftp://ftp.ibiblio.org/pub/linux/distrib ... /Metalinks
rsync://ibiblio.org/pub/linux/distribut ... /Metalinks
WarraWarra
Sagely Hen
 
Posts: 1989
Joined: Wed Jan 03, 2007 21:01
Location: 31.324270, -113.585511

Re: Automate mirror usage and checksums for ISO downloads

Postby Nicolas » Sun Nov 04, 2007 18:02

WarraWarra wrote:If anyone wants to or have the time to make new metalinks or update the current ones for mostly new versions of SL or all versions just shout as the ibiblio.org server now has a metalink folder so we can post them on there , redistribute them from there etc etc . Just buzz me / send me the links and I will download / post them there or something like this.

They're already here: http://www.metalinker.org/samples/sabayon/
Nicolas
Baby Hen
 
Posts: 15
Joined: Sun Apr 15, 2007 23:09

Re: Automate mirror usage and checksums for ISO downloads

Postby twanj » Sun Nov 04, 2007 22:32

Nicolas wrote:
WarraWarra wrote:If anyone wants to or have the time to make new metalinks or update the current ones for mostly new versions of SL or all versions just shout as the ibiblio.org server now has a metalink folder so we can post them on there , redistribute them from there etc etc . Just buzz me / send me the links and I will download / post them there or something like this.

They're already here: http://www.metalinker.org/samples/sabayon/


I think it's great to have them on ibiblio.org but still link to metalinker.org on the download page, like things are now.
twanj
Baby Hen
 
Posts: 8
Joined: Sun Apr 22, 2007 19:35


Return to Artwork and Development Suggestions

Who is online

Users browsing this forum: No registered users and 3 guests

cron