A great Sabayon feature could be: Scientific Computing Mode

[archwndas]

Hi there,
I would just like to mention about a feature no Linux Distro has bothered to do up to now. Quite frequently people who work in scientific computing want to benchmark their codes. Those are quite memory and CPU intensive codes and there is a lot of competition out there. So, even the "last drop" of CPU and memory resources could make a difference.

Many - many people would love that one. Trust me. I know quite a lot of them and we were dissapointed nobody offered something like that in the past. All one has to do is install Gentoo and try to add a new grub-menu boot options to boot with a specific kernel and then turn off with (rc-update stop) all daemons that are not desired. This was our only option.

I am suggesting here a special mode of Sabayon which could be selected during boot or logout whatever, where a lot of the background running daemons (powersave e.t.c.) stop, a lot of recourses
are freed and a very light and fast window manager gets loaded such that we do not run the benchmark on the slow scroling framebuffer.

Perhaps in the future if this feature is added at the boot loader then one could might consider even a special kernel configuration tweaked for (UP/SMP) scientific computing.

So, what do you think, is it feasible?

SuSE provides a safe mode boot option. One could modify that to our purpose.

1. Load a special tweak kernel,
2. turn of all running daemons such that all the do not waste any CPU resources
3. load a very light window manager: twm, icewm, or something similar.

Thanks for the attention,
Archwn.

[Stupot]

I'd rather keep Sabayon our bleeding edge distro and there are not enough devs to make it both.

[R3MF]

agreed.

[archwndas]

I'd rather keep Sabayon our bleeding edge distro and there are not enough devs to make it both.

Guys please read carefully. I never said to change Sabayon. I just said to add an option at the grub loader such that the user can either boot with the classical Sabayon mode or choose the scientific computing mode. If the user chooses the scientific computing mode then Sabayon boots with a kernel optimized for scientific computing and with all (except the essential) running daemons off. Is it so hard? I told you that SuSE has a safe mode boot option in the grub boot menu. I hope you understood what I mean.

[zouzou85]

archwndas, I am not sure I understood you right, but if you mean that you would like to see a "safe boot" option like in many other distros (SuSE, Ubuntu, etc..) i'd agree with you. since sometimes one needs to do heavy work, and many daemons which take away resources from the main process are not needed.
So, yeah, i like your idea!
hope i understood you right

[archwndas]

archwndas, I am not sure I understood you right, but if you mean that you would like to see a "safe boot" option like in many other distros (SuSE, Ubuntu, etc..) i'd agree with you. since sometimes one needs to do heavy work, and many daemons which take away resources from the main process are not needed.
So, yeah, i like your idea!
hope i understood you right

Exactly. That's my point. But, it will not be a safe mode, but a scientific computing mode since:

1) It may boot a completely different kernel tweaked such as it allows user space applications to
run as fast as possible.

2) It turns all daemons off and loads the lightest but still effective(icewm) window manager.

I do not think that this will be hard for those experts-gurus who built Sabayon. It is a great distribution
and I admire their work. Why shouldn't we motivate them to help scientific computing guys such as
Sabayon becomes popular among this group of people as well. There is really no Linux distribution we
can use for our purposes.

Once I installed Puppy Linux 2.11, Gentoo with gentoo-sources and gentoo with ck-sources and
contacted the ck-sources author to ask for tweaks in kernel such that we have a kernel optimized for
performance of our applications. Do you know what the result was:

my codes which run under Gentoo booted with ck-sources had the same runtime performance as
those with plain gentoo-sources. The code which run under puppy linux outperformed everything
else by 10%. Yes you heared right. The runtime performance of the same code under puppy 2.11
was by 10% faster than the same code under gentoo-sources Gentoo linux. Have in mind that when
I booted Gentoo I had removed with the rc-update all the daemons I didn't need including ssh e.t.c.
Puppy Linux kernel is not compiled for 486. So what does really make the difference?

I would like to note that the code was a calling a precompiled .so library for solving large sparse linear
systems which is the most important kernel in plenty of scientific computing applications.

Best,
Archwn.

[zouzou85]

sorry to ask, but what are some examples of scientific computing? i mean, could you tell me how you use a scientific computuation app, how it works, and what would be the outcome you would like of the app?
I glanced over wikipedia for scientific computation, but it just went over my head.
if it is uncharted territory, i would definitely love to see this distro chart and map it for the rest, but we got to keep in mind those who tried unsuccessfully.

zouzou85

[archwndas]

sorry to ask, but what are some examples of scientific computing? i mean, could you tell me how you use a scientific computuation app, how it works, and what would be the outcome you would like of the app?
I glanced over wikipedia for scientific computation, but it just went over my head.
if it is uncharted territory, i would definitely love to see this distro chart and map it for the rest, but we got to keep in mind those who tried unsuccessfully.

Scientific computing involves a lot of different areas of science were the people use computers to simulate and to understand what is going on in real-life problems. A wide area is that of people who
are working with the Finite Element Method or Finite Volume Method (FEM, FVM). The applications are quite a lot: aerodynamic simulations (optimal wing, aircraft design, formula 1) crash tests of cars (performed in a PC), biological flows, fuel cells, semiconductor device simulations, electromagnetics, scattering theory, wave propagation, structural mechanics , just to name a few.
The other group of people are working with molecular simulations. There they use molecular dynamics MD, Monte-Carlo algorithms to predict and study the physical properties of arbitrary polymers and several other materials, how polymers interact with drags device new polymers , dendrimers which are drug curriers e.t.c.

All those computer programs are memory and CPU intensive and may run for several days even months. There exists dedicated servers for those, but the developers of those codes may not have access to those servers. When you are designing a software and you want to optimize your methods
and algorithms even your implementation, you need accurate timings. These are not easy to achieve if in the background a lot of undesirable daemons are running. All of you know that GNOME and KDE consume a lot of memory compared with other light window managers. Memory in Laptops which are
used frequently for developing nowdays is limitted to at most 2GBs if I am not wrong. Suppose that you want to see how a code scales in terms of memory consumption (sparse direct linear solver) and plot some graphs. In those cases you need to free as much memory as possible such that it can be used by your application. There you definately need the lightest window manager.

I do not know any Linux Distro designed especially to provide all the resources to the user and consume as less as possible for other stuff. I do not know any special Linux Kernel tweaks which would result in optimum performance of user space applications. It would be wonderful if Sabayon could tackle this and offer a special kernel (obtained after lots of benchmarking) along with a configuration loaded at boot time for people who would like to boot and run their scientific computing applications either for benchmarking during the development process or for collecting results.

If you are interested I could provide a sample code such that you can run and test the runtimes in different loads of your sabayon installation.

Best,
Archwn.

[zouzou85]

That sounds interesting!!
I would definitely love to see a kernel devoted to simply be optimally performant, especially for heavy duty work.
but what do you think of the sabayon core installation. (in this thread http://planet.sabayonlinux.org/?p=50)
I know the core installation is not the tweaked kernel that's you are asking for, but how about it for a start.
But i would definitely want to see a specially tweaked kernel just for optimum performance.

i know that it is not what you are talking about, but just give it a try and let us know what you think of it!

Edit: and yeah, please send me the code (with a little bit of instructions and what to look for as outcome). sound interesting to test!

[archwndas]

That sounds interesting!!
I would definitely love to see a kernel devoted to simply be optimally performant, especially for heavy duty work. But what do you think of the sabayon core installation (in this thread http://planet.sabayonlinux.org/?p=50) ? I know the core installation is not the tweaked kernel that's you are asking for, but how about it for a start. But i would definitely want to see a specially tweaked kernel just for optimum performance. I know that it is not what you are talking about, but just give it a try and let us know what you think of it!

Edit: and yeah, please send me the code (with a little bit of instructions and what to look for as outcome). sound interesting to test!

Sabayon Core:
This is a must. Why should I install so many things in my linux / partition when I need only a few? Sabayon takes as much as 10GB space. That is too much. Even SuSE with all the crappy stuff in there usually takes more or less 3GB of space. I would like to mention here that Sabayon should include more window managers. Not only E16 and FluxBox as alternatives. FluxBox is one of the worst Window Managers out there. No easy access to desktops (compare with IceWM) not easy customization of menus. My favorite light-weight window manager is IceWM and it has also very nice themes. Once should also cosnsider the JWM which is used in PuppyLinux. They have a very nice icon-theme.

Back to Sabayon Core, I believe that they should also provide an option of tweaking the scripts in the default and/or boot runlevel. I may do not need network or alsasound or xfs or vixie-cron or all the other stuff in there. If they can do that then this is allright. I could just go on and install sabayon on a different partition as light as possible with the Window Manager of my preference and with the init-scripts of my preference. No Gnome no KDE no funcy stuff. Just the lightest Sabayon that can ever be installed with emacs TeTeX, LabPlot and some compilers. How can I tell the main developer of Sabayon to include IceWM TeTeX and LabPlot in the 3.4 DVD? Then what is left is an optimized kernel. Suppose we find that after testing several kernel sources and changing several kernel parameters and recompiling and benchmarking. Quite a task but definately interesting and enjoyable task. Linux Kernel provides an option to boot a new Linux kernel without rebooting. So suppose that we have our optimized kernel at /boot and we log out GNOME (standard Sabayon) and choose from the KDM or GDM options to boot to the Scientific Computing mode. The coresponding scripts could turn all redundant scripts off and boot (without rebooting) the optimized Linux kernel stored in /boot while entering IceWM or someother light weight Window Manager (even quiting gdm to free resources). This is by far something nobody has ever thought of implementing and Sabayon will be the first to do.

I will give Sabayon Core a try as soon as the final 3.4 DVD is out. It takes almost 1 day to download Sabayon Linux DVD. But how can I turn off all the corresponding scripts (/etc/init.d/script stop)? I do not have time to play and recompile different sabayon kernels. I do not know what the implications will be of installing a Sabayon new kernel.

Regarding the code send me a private message with your email address and I will send it over as an attachment along with tips and tricks ...

Regards,
Archwn.