binary repository

[random guy]

yesterday i was looking for a good web designing program and basically since i didnt know which ones were good until i tried them i had to compile each one and then see if i liked it and then uninstall ones i didnt... this can be a real hassle since it only takes me one min to decide if i like the app or not but it takes me a lot more than one min to compile.

i know i am just another guy complaining about compiling but the way i see it i want binary packages for testing and then just compile the one i want from source not all the ones i decide i didnt want and decide to uninstall them anyway.

since the sabayon dvd has so many packages on it (a lot more than the mini) it would be a good idea to have a binary repository of the packages on the dvd so that people can download and use them. sabayon already made the binaries for the disk why not just put them online one by one.

i am all for compiling from source for something i want but not for something i dont want.

[cvill64]

a) bandwidth fees would be huge if we posted everything
b) we're thinking about it already
c) there is a binary mirror out there, on larrythecow links I think, check it out

[random guy]

c) there is a binary mirror out there, on larrythecow links I think, check it out

ya i think you have mentioned that before but i wasnt able to locate any such link on larrythecow.com or .org. .org seems to be nothing and in the place on .com where you would suspect some information about a mirror http://larrythecow.com/?cat=43 it doesnt have a link to a binary one.

a) bandwidth fees would be huge if we posted everything

ya i was afraid of that.

[whilo]

i've thought about binary hosts for gentoo very often and i'm convinced that this is the biggest feature missing in gentoo. debian is still much better if you want a simple stable system with a big repository and regular updates.

but gentoo could provide the base for all other distributions. sabayon for example proves that, as it is a specialized gentoo distribution for average desktop users and it has a great design and usability advantage over vanilla gentoo. we could solve the big problems with the old and crappy rpm distros like fedora/redhat or suse/opensuse, which still have software management problems today and miss a hell lot of software.
i don't think they would ever switch over to a gentoo base system and build their binary system around/ on top of it, but distributions like them, e.g. sabayon can do it.

a big advantage would be, since the base system of the different modified gentoo systems is gentoo, that you could switch from one system to the other without a reinstall. if i want to switch from sabayon to, lets say, the x-distribution for hardened systems i just have to reset the /etc/make.conf file, reinstall world (which takes a lot of time but is not a big issue with bin packages) and adjust some configurations.
and you could do it the other way around, which is really interesting for sabayon. gentoo users could use a fitting make.conf for sabayon bin packages and could switch over to sabayon more or less, if they want an average desktop system but still keep their old system in one way or the other (no backups needed and ( almost )no reconfiguration).
i hate the distro chaos , although i believe many specialized distros are much better than one monolithic system (e.g. xp or mac).

the biggest problem for me is that the very base of all distros is not the same. this could be achieved with gentoo very easily. sabayon is an example for that. an advanced gentoo user can built his own customized distribution (livecd) on top of gentoo.

still most users prefer binary distributions, because they don't want to compile all day for a running system (which is the case if you have older hardware and do daily updates). this is not good for me, too, since it has an impact on hardware. gentoo is more expensive than other distros in many ways and therefore still a distro for hobbyists.

binhosts are the only solution. for gentoo itself it is senseless, since gentoo is not customized and would loose its advantage with bin hosts. it serves as a skeleton distribution/meta distribution for specialized distros.
but on these specialized distros, binhosts make a lot of sense, they are necessary. otherwise they are no competition (i don't mean that in bad way) to existing specialized distributions on other bases (rpm or debs for example). arch linux is an interesting mixture, which is quite similar to gentoo, but provides binary packages as well, and is growing rapidly.

furthermore the fear of getting an enormous bandwith problem is on the other side the probability of an enormous growth of the sabayon user base with bin hosts. sabayon could be the distro 'to go'

to cut it short, we need a binhost . i've done some research today and i've found out, that i could get an provider (http://www.hetzner.de) with 1 terrabyte traffic included. if i get more traffic it will be reduced to a 10 MBit connection (this means it's like all inclusive with a little drawback).
the machine would be 3700+ amd athlon, one gig ram, 160gb hd (raid 1?) debian/suse (no gentoo, but that's no idealogical problem for me). it would be in germany at nürnberg and the cost would be 39€ a month. i don't know about the existing infrastructure of sabayon, but maybe we could set another mirror up ( in the states, maybe? ).
i could try to get in contact with my university (uni heidelberg), too. but i don't know if they support it. in the beginning we could do weekly updates from the sabayon built machines, updating to a sane linked system. this should be enough to test the impact of the binhosts. maybe the cost could be shared somehow, since i'm only a student.
what do you think?

cheers,
whilo

[cvill64]

We have 2 TB of transfer per month, but if you start with a bin host....trust me, bandwidth would be enormous. A little server like that really wouldn't do it in the long run and binaries are just in short....inferior. I would suggest a gentoo binhost for a company no problem, because all their systems are usually the same and they don't want to compile all day long. There are also other complications of getting the man power to upkeep it and many others. we shall discuss this more among ourselves but feedback like yours is much appreciated

[random guy]

well you already made the binaryies with the live dvd. maybe if the live dvd in inserted in the drive it can do an "install additional packages" thing and install new programs from the dvd enviorment to the base system. not sure if the most resent version of the installer allowed for the user to select individual packages for installation but when/if that happens or even not it would be a good way to update software, reinstall removed software, maybe even update the system (if the user only wants to update parts of the system not all, umm not sure why someone would want this though).

anyway it seems like a reasonable solution: no more bandwidth or server costs, probably wouldnt be that hard to set up, and it would most importantly work. although not be as flexible as an actual binhost but it could be helpful.

[cvill64]

they can already do this. Just follow the recover gcc tutorial and then just change gcc for w/e

[whilo]

A little server like that really wouldn't do it in the long run and binaries are just in short....inferior. I would suggest a gentoo binhost for a company no problem, because all their systems are usually the same and they don't want to compile all day long.

Sorry, but I don't think I can agree here. You are absolutely right about Gentoo, binary packages are nonsense. But I'm not talking about Gentoo here, I'm talking about Sabayon or other binary(!) derivates of Gentoo. They are like many other (major) distributions binary, they only use Gentoo for easier software management.
But if you consider Sabayon as a binary distribution with a very static make.conf (CHOST,CFLAGS,USE environment) binary packages make sense, because:

- they can be used by every sabayon user, who didn't change the make.conf or the make.profile or anything else in the environment (most desktop users won't do that), which means that they are actually Gentoo users. We only have two build profiles at the moment: i586/x86 and amd64/x86_64.

- every major binary distribution has binary online updates today. I don't think that doing a dvd update every two month is a very good approach. It is working, of course, and you can ship the packages with a new consistant configuration and you have a good BitTorrent solution, but it is not working for small software updates, consistant security or bug fixes or for additional software installation. Debian for example needs 3 dvds for all packages... Most likely advanced linux hobbyists will do gentoo source updates themselves, but not the average Desktop user. They even don't want to do a 2 month software dvd update. They need an automatic GUI-Software-Updater (SuSE, RedHat, Debian, Windows, Mac OS X,...) and they don't want to wait three days for a complete update.

Almost any distributional need could be satisfied by Gentoo derivates today. There is no need for any other package management tool anymore, since Portage can beat them all.
This is not about flaming or underestimating the great APT tool (or others), but it is about the idea of unifying Software distributions with an almost universal software management tool like Portage. Portage in combination with special configurations and binary repositories for several binary Gentoo derivates would be a solution to many Linux/BSD/Solaris problems today. Do the general package management stuff for gentoo ebuilds together and customize the binary build environment for your distribution afterwards. In my opinion this is what Sabayon does, isn't it?

You can see the big advantage of that directly: You can extend/change your binary distribution easily with source builds from gentoo and satisfy your needs with a big gentoo base repository (while keeping your binary package updates from your customized distribution).

The conclusio here is, that binary packages for Gentoo derivates are good!

The problem are not the packages themselves, but distributing them. Most likely you are right here and I have no idea about actual server needs for a binhost or other solutions.

My idea was:

Start the project; wait for the impact; try to find as many mirrors as possible (and find a better server solution later maybe).
This impact would show that there is a great need for binhosts and prove me right . Just because there are problems doesn't mean that it is not a good thing.

But other solutions came to my mind, too. What about using P2P? Something like BitTorrent or Edonkey? I know that sounds really strange for package updates, but it is a real community solution. The advantages are, if we find a working technique:

- scalability
- prove of concept and solution for almost every other Gentoo derivate (even small home brewn)
- makes us a competitive distribution with "one" step

The big question is: How can you get a reliable and relatively fast p2p solution (it shouldn't take longer to download the file, then to build it from source using a fast Gentoo mirror)?
Portage gives us the FETCHCOMMAND environment variable, where we can set customized solutions to get our packages. We could create a background daemon, BitTorrent for example, which fetches the update and distributes it back afterwards.
I don't know how to do the actual packaging. For the beginning I would think of weekly forks out of the Sabayon development repository, which build cleanly and don't have any revdep-rebuild issues. Then package all updated packages together and finally setup a torrent tracker, which can be reached by the update daemons of the clients. Maybe the clients could decide themselves which bin packages of the torrent they choose.

I don't know how much manpower is needed for automatical build scripts on the Sabayon development server. Could you precise where the problems are here, please?

I understand that there can be stability problems and even hardware, xorg, ... issues. Maybe we could do a fork out of the released Sabayon version, e.g. 3.1, and then do an "emerge world -u" weekly? Maybe fix some critical bugs, too. But since they are already fixed Gentoo upstream most likely, it wouldn't be as much work as on a "normal" distribution. Probably some manpower comes back, once Sabayon gets more popular.

Personally I believe Sabayon is better than all other distributions, the only drawback is the source building. (By the way thank you very much for: 1. a very nice design (the best on all distros) 2. the best LiveDVD I've ever seen 3. the best Gentoo installation support( hardware detection/preconfiguration ). Without the long boot times and the missing "-mno-tls-direct-seg-refs" CFLAG I would use it today. I would even drop the non provable build optimisations )

I know this is a very vague concept, but I would like to discuss that here. I hope you like to do it here, too. The Gentoo forums are in my opinion not the real target for this discussion, although they might be interested.

There are also other complications of getting the man power to upkeep it and many others. we shall discuss this more among ourselves but feedback like yours is much appreciated

What do you mean with "among ourselves" ? Do you mean we should discuss that in a private chat or do you mean you would like to discuss that in the sabayon leaders group only?

Cheers,
whilo

[wolfden]

Right now it's tough just distributing SabayonLinux isos yet alone binaries. You can get the packages you need through portage most of the time. So you have to compile, what is the problem? You can always download binaries on your own. P2P isn't going to work cause people don't have patience to wait. Some don't even know what P2P is yet alone configure it, port forward and use it.

SabayonLinux needs to stay focused on the operating system and supporting it. SL needs the support of the people in order to continue development as it is.

We should be happy with what we have - it works great.

[lxnay]

Apart from hosting's problems, we would have some other problems:

1. miniEditions: since minis don't come with GNOME deps, we can't produce a binary package that works on both DVD edition and those ones.
2. we'd need a lot of extra man power.
3. we need money and (at least me) get paid for doing that, since I already work 8hrs/day in the "real" life.
4. Ref. point 2, we'd need to create an extra layer on top of portage.

To sum up, we could do everything but, we need time, and time = money, and without money you can't live.