Seeding Sabayon torrents

Discussion in general that pertains to Sabayon Linux - Must Pertain to Sabayon Linux

Moderator: Moderators

Post Reply
svantoviit
Old Dear Hen
Posts: 724
Joined: Sun Feb 28, 2010 17:55
Contact:

Seeding Sabayon torrents

Post by svantoviit » Wed Apr 17, 2013 12:08

Since I have rtorrent running all the time in the background I decided to seed all Sabayon ISO images available from http://torrents.sabayon.org/.
With the new monthly releases (see lxnay's blog for more information: What's cookin' on the BBQ | On The Other Hand) the number of the files grew noticeably (32 at the moment).
I'm a lazy dog, so I created a short bash script, which fetches new torrent files from the fastest mirror and puts them into my rtorrent watch directory. I though it could be useful for someone else too, so here it is:

Code: Select all

#!/bin/bash

# set watch directory
WATCH=/home/$USER/Downloads/torrents/watch

# get the download url
URL=`wget -S --spider http://packages.sabayon.org/r/iso/ 2>&1 | awk '/following/ {print $2}'`

# fetch the torrent files
wget -erobots=off -r -nd -np -nc -c -A.torrent -P $WATCH $URL
I'm not much into coding and do not understand the bittorrent protocol, so there's for sure a better way to do it. Please, let me know.

To avoid running the script manually I run it using cron as regular user at the beginning of the month.

lxnay
Land Owner
Posts: 3596
Joined: Thu Oct 13, 2005 23:16
Location: Italy
Contact:

Re: Seeding Sabayon torrents

Post by lxnay » Wed Apr 17, 2013 15:12

Or you could just use a simple rsync cronjob that fetches the torrents. Much more reliable.
Image
Join us on IRC (chat.freenode.net #sabayon or WebChat)
Submit bugs to our Bug Tracker
Follow me on Twitter
Add me on Facebook
Add me on Google+

svantoviit
Old Dear Hen
Posts: 724
Joined: Sun Feb 28, 2010 17:55
Contact:

Re: Seeding Sabayon torrents

Post by svantoviit » Thu Apr 18, 2013 8:52

Thanks for the tip. This does the job too:

Code: Select all

rsync -rqz --include='*.torrent' --exclude=* rsync://na.mirror.garr.it/sabayonlinux/iso/{,monthly/} /home/$USER/Downloads/torrents/watch
And for those interested here's the list of rsync mirrors one can choose from:

Code: Select all

rsync://mirrors.coopvgg.com.ar/sabayon/
rsync://gd.tuwien.ac.at/linux/sabayonlinux/
rsync://rsync.belnet.be/sabayon/
rsync://sabayon.c3sl.ufpr.br/sabayon/
rsync://mirror.freelydifferent.com/sabayon/
rsync://ftp.fsn.hu/linux/sabayon/
rsync://na.mirror.garr.it/sabayonlinux/
rsync://ftp.kddilabs.jp/sabayonlinux/
rsync://ftp.tsukuba.wide.ad.jp/sabayon/
rsync://ftp.nluug.nl/sabayonlinux/
rsync://ftp.SURFnet.nl/sabayonlinux/
rsync://ftp.rnl.ist.utl.pt/pub/sabayon/
rsync://mirror.yandex.ru/sabayon/
rsync://mirror.clarkson.edu/sabayonlinux/
rsync://mirror.umd.edu/sabayonlinux/
rsync://sabayon.mirror.ac.za/
rsyncwget
1st run real 0m5.553s
user 0m0.125s
sys 0m0.256s
real 0m5.750s
user 0m0.079s
sys 0m0.271s
2nd runreal 0m1.161s
user 0m0.107s
sys 0m0.086s
real 0m0.769s
user 0m0.018s
sys 0m0.018s
Both wget and rsync perform at about same speed, wget seems to be a tiny bit faster overall…

User avatar
sabayonino
Sagely Hen
Posts: 3190
Joined: Sun Sep 21, 2008 1:12
Location: Italy
Contact:

Re: Seeding Sabayon torrents

Post by sabayonino » Thu Apr 18, 2013 20:06

or :mrgreen:

All torrents

Code: Select all

rsync  --include="*.torrent" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ /where/you/want
Only 64bit torrents (or x86)

Code: Select all

rsync  --include="*_amd64*.torrent" --exclude=*  -avz --delete-after rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ /where/you/want/

Sync ISOs

All ISOs

Code: Select all

rsync  --include="*.iso" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ /where/you/want
Only 64bit (or x86) ISOs

Code: Select all

rsync  --include="*_amd64*.iso" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ /where/you/want

For seeding set your client to auto-load your torrent folder

Script this in your cronjob and consider (optionally) to remove "-v" (verbose) option and set /dev/null for stderr e/o stdout (2>&1 >/dev/null)



regards
[Che Cos'è Il Calcolo Distribuito (BOINC)

BOINC ready ! Sabayon+BOINC = BILD ,my Sabayon spin :cyclops: - Ready to crunch for the Science everywhere :)

Ryuno-Ki
Old Dear Hen
Posts: 483
Joined: Mon Feb 27, 2012 22:15
Location: Southern Germany
Contact:

Re: Seeding Sabayon torrents

Post by Ryuno-Ki » Thu Apr 18, 2013 23:39

sabayonino wrote:*snip*
set /dev/null for stderr e/o stdout (2>&1 >/dev/null)
As far as I understand bash syntax, it must be

Code: Select all

command-foo > /dev/null 2>&1
We had this topic on IRC a few days ago.
What happens is the following:
Set STDOUT(= 1) 1> (or shorthand: >) to /dev/null and then bind STDERR (= 2) to the same target as 1: 2>&1.
German Wiki translator ◆ Want to contribute on the wiki? Ask the Bureaucrats!
Please, mark solved threads ◆ Posts unread and unansweredPackage requests

svantoviit
Old Dear Hen
Posts: 724
Joined: Sun Feb 28, 2010 17:55
Contact:

Re: Seeding Sabayon torrents

Post by svantoviit » Fri Apr 19, 2013 6:40

sabayonino wrote:

Code: Select all

rsync  --include="*.torrent" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ /where/you/want

Script this in your cronjob and consider (optionally) to remove "-v" (verbose) option and set /dev/null for stderr e/o stdout (2>&1 >/dev/null)
You forgot about the standard releases, what you fetch are only the monthly releases. Noticed the expansion in my solution?

I would be cautious about the --delete-after option. It will delete all the other torrents you are watching!

Code: Select all

$ rsync  --include="*.torrent" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ .

receiving file list ... done
./
Sabayon_Linux_13.04_amd64_GNOME.iso.torrent
Sabayon_Linux_13.04_amd64_KDE.iso.torrent
Sabayon_Linux_13.04_amd64_MATE.iso.torrent
Sabayon_Linux_13.04_amd64_SpinBase.iso.torrent
Sabayon_Linux_13.04_amd64_Xfce.iso.torrent
Sabayon_Linux_13.04_x86_GNOME.iso.torrent
Sabayon_Linux_13.04_x86_KDE.iso.torrent
Sabayon_Linux_13.04_x86_MATE.iso.torrent
Sabayon_Linux_13.04_x86_SpinBase.iso.torrent
Sabayon_Linux_13.04_x86_Xfce.iso.torrent
Sabayon_Linux_CoreCDX_13.04_amd64.iso.torrent
Sabayon_Linux_CoreCDX_13.04_x86.iso.torrent
Sabayon_Linux_HardenedServer_13.04_amd64.iso.torrent
Sabayon_Linux_HardenedServer_13.04_x86.iso.torrent
Sabayon_Linux_ServerBase_13.04_amd64.iso.torrent
Sabayon_Linux_ServerBase_13.04_x86.iso.torrent

sent 363 bytes  received 3403974 bytes  972667.71 bytes/sec
total size is 3401580  speedup is 1.00

$ ls
Sabayon_Linux_13.04_amd64_GNOME.iso.torrent
Sabayon_Linux_13.04_amd64_KDE.iso.torrent
Sabayon_Linux_13.04_amd64_MATE.iso.torrent
Sabayon_Linux_13.04_amd64_SpinBase.iso.torrent
Sabayon_Linux_13.04_amd64_Xfce.iso.torrent
Sabayon_Linux_13.04_x86_GNOME.iso.torrent
Sabayon_Linux_13.04_x86_KDE.iso.torrent
Sabayon_Linux_13.04_x86_MATE.iso.torrent
Sabayon_Linux_13.04_x86_SpinBase.iso.torrent
Sabayon_Linux_13.04_x86_Xfce.iso.torrent
Sabayon_Linux_CoreCDX_13.04_amd64.iso.torrent
Sabayon_Linux_CoreCDX_13.04_x86.iso.torrent
Sabayon_Linux_HardenedServer_13.04_amd64.iso.torrent
Sabayon_Linux_HardenedServer_13.04_x86.iso.torrent
Sabayon_Linux_ServerBase_13.04_amd64.iso.torrent
Sabayon_Linux_ServerBase_13.04_x86.iso.torrent

$ touch {1..10}.torrent

$ rsync  --include="*.torrent" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ .

receiving file list ... done
./
deleting 9.torrent
deleting 8.torrent
deleting 7.torrent
deleting 6.torrent
deleting 5.torrent
deleting 4.torrent
deleting 3.torrent
deleting 2.torrent
deleting 10.torrent
deleting 1.torrent

sent 59 bytes  received 578 bytes  424.67 bytes/sec
total size is 3401580  speedup is 5340.00
As Ryuno-Ki pointed out the right syntax for the redirection of the output is command-foo > /dev/null 2>&1 or

Code: Select all

command-foo &> /dev/null
This will redirect both the standard output and the error messages. I prefer the -q option which suppresses all but the error messages.

User avatar
sabayonino
Sagely Hen
Posts: 3190
Joined: Sun Sep 21, 2008 1:12
Location: Italy
Contact:

Re: Seeding Sabayon torrents

Post by sabayonino » Fri Apr 19, 2013 14:48

svantoviit wrote: I would be cautious about the --delete-after option. It will delete all the other torrents you are watching!

Code: Select all

$ rsync  --include="*.torrent" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ .

receiving file list ... done
./
Sabayon_Linux_13.04_amd64_GNOME.iso.torrent
Sabayon_Linux_13.04_amd64_KDE.iso.torrent
Sabayon_Linux_13.04_amd64_MATE.iso.torrent
Sabayon_Linux_13.04_amd64_SpinBase.iso.torrent
Sabayon_Linux_13.04_amd64_Xfce.iso.torrent
Sabayon_Linux_13.04_x86_GNOME.iso.torrent
Sabayon_Linux_13.04_x86_KDE.iso.torrent
Sabayon_Linux_13.04_x86_MATE.iso.torrent
Sabayon_Linux_13.04_x86_SpinBase.iso.torrent
Sabayon_Linux_13.04_x86_Xfce.iso.torrent
Sabayon_Linux_CoreCDX_13.04_amd64.iso.torrent
Sabayon_Linux_CoreCDX_13.04_x86.iso.torrent
Sabayon_Linux_HardenedServer_13.04_amd64.iso.torrent
Sabayon_Linux_HardenedServer_13.04_x86.iso.torrent
Sabayon_Linux_ServerBase_13.04_amd64.iso.torrent
Sabayon_Linux_ServerBase_13.04_x86.iso.torrent

sent 363 bytes  received 3403974 bytes  972667.71 bytes/sec
total size is 3401580  speedup is 1.00

$ ls
Sabayon_Linux_13.04_amd64_GNOME.iso.torrent
Sabayon_Linux_13.04_amd64_KDE.iso.torrent
Sabayon_Linux_13.04_amd64_MATE.iso.torrent
Sabayon_Linux_13.04_amd64_SpinBase.iso.torrent
Sabayon_Linux_13.04_amd64_Xfce.iso.torrent
Sabayon_Linux_13.04_x86_GNOME.iso.torrent
Sabayon_Linux_13.04_x86_KDE.iso.torrent
Sabayon_Linux_13.04_x86_MATE.iso.torrent
Sabayon_Linux_13.04_x86_SpinBase.iso.torrent
Sabayon_Linux_13.04_x86_Xfce.iso.torrent
Sabayon_Linux_CoreCDX_13.04_amd64.iso.torrent
Sabayon_Linux_CoreCDX_13.04_x86.iso.torrent
Sabayon_Linux_HardenedServer_13.04_amd64.iso.torrent
Sabayon_Linux_HardenedServer_13.04_x86.iso.torrent
Sabayon_Linux_ServerBase_13.04_amd64.iso.torrent
Sabayon_Linux_ServerBase_13.04_x86.iso.torrent

$ touch {1..10}.torrent

$ rsync  --include="*.torrent" --exclude=*  -avz --delete-after  rsync://na.mirror.garr.it/sabayonlinux/iso/monthly/ .

receiving file list ... done
./
deleting 9.torrent
deleting 8.torrent
deleting 7.torrent
deleting 6.torrent
deleting 5.torrent
deleting 4.torrent
deleting 3.torrent
deleting 2.torrent
deleting 10.torrent
deleting 1.torrent

sent 59 bytes  received 578 bytes  424.67 bytes/sec
total size is 3401580  speedup is 5340.00
I know. but after one more month you keep many torrents that will be outdated (for monthly or daily or stable releases ) :roll:

you can configure your torrents folder as you want :bom:
[Che Cos'è Il Calcolo Distribuito (BOINC)

BOINC ready ! Sabayon+BOINC = BILD ,my Sabayon spin :cyclops: - Ready to crunch for the Science everywhere :)

Post Reply