Page 1 of 1

Something's broken - Repairing the system [Solved]

Posted: Sat Aug 21, 2010 14:10
by amoebios
(bear with me, i don't have other hobbies.... or anything to do)

Even if this incident shall drive me away from Sabayon, i figured i'd start a thread on how to repair your system when things go wry, since i've already gathered some experience in that direction. Eventually, a wiki article can be created from the contents within this thread. But most-importantly; help me fix my stuff!

What, how and why
The story behind what made me write this and how i got where i am: August 15th, 2010. Some time around morning i had this idea of a simple boot splash. My initial idea of just having the percentage of the booting progress displayed in big bold letters didn't work out (i figured i'd need a different boot engine = too much work), so i settled for an even simpler design, and, to my surprise, it worked really well: http://amoebios.wikispaces.com/splash.loading. Then i wanted to try something else, and, to not having to reboot the system every time i wanted another way to test the splashes. There is a tool for that, but getting it to work requires some patience. i remembered reading a mention about qemu on this forum, so i searched a bit and found the command "qemu -hda sda" (Don't do this, obviously. Add "-snapshot", at least.), which seemed to work alright. After trying to restart the virtual system it seemed to be stuck, so i closed the emulator window. Everything seemed fine and i kept doing usual things, like searching files, editing graphics and reading man pages. i simulated booting with qemu several times, but it printed some errors of which i thought were due to the "fake booting", where software tries to access hardware that isn't really there. That's what i thought. i don't understand emulation. Anyway, the errors prevented me from viewing the silent boot splash, so i restarted my laptop. On the next startup the system failed to start the X server, went to the command line and printed "(none)", where "acer-sabayon" should be printed.

Backing up
Now, before attempting anything, you should save what's left to save with a LiveCD, for example. i used Knoppix, the mother of all LiveCD's (the LiveUSB i used to install Sabayon was long overwritten). On a default Sabayon install you first need to make the filesystem accessible. At this point i should mention that you will need to use the command line. Open a terminal. (the $ at the beginning of a line indicates a normal user, while a # indicates root. You don't need to type these.)

Code: Select all

$ su
(type your password or just press ENTER when in a liveCD)
# cd /mnt
# mkdir lvm
# vgscan --mknodes
# lvchange -a y /dev/VolGroup00/LogVol00
# mount /dev/VolGroup00/LogVol00 /mnt/lvm
(You can view what logical volumes you have with pvs and lvdisplay. More information on logical volumes is here: http://wiki.sabayon.org/index.php?title ... _Mount_LVM)

If everything went as planned you can now access the logical volume at /mnt/lvm and copy the contents to your backup media. (most-likely an external USB drive or a second disk)


The Repairing begins - Restoring passwords
A useful tool for repairing a system is chroot. Okay, i only ever used it to restore passwords, but theoretically it could be useful for lots of other things, like.... since it.... okay, restoring passwords is probably the most useful thing, so let's stay with that. Here's how:
First, you gotta mount the system. In our case, we have already mounted it on /mnt/lvm and we can leave it there. You still have the terminal open, right?

Code: Select all

# chroot /mnt/lvm /bin/bash
/# passwd
(type a new password)

Bug-hunting
To fight bugs you need to find them. In log files, for example. In Sabayon there's a toolset (a script, technically) called
Sabayon Live Tools that can be installed with the command

Code: Select all

# equo update (we need to update first)
# equo install sabayonlive-tools

now, you can launch it with
# sabutil menu
More documentation is here: http://forum.sabayon.org/viewtopic.php?f=57&t=15530

Reporting
If you have a bug cornered and know what to report do so at http://bugs.sabayon.org/ .

If all else fails
"Google is your friend", they say. Now, i never met or talked to Google in person, but i do find its use helpful at times.

"Halp!"
Equipped with this knowledge i attempted to fight back the aggressor (the evil bugs), but, alas, i'm outnumbered, need help! (at:home) (health:1) (armor:0) (w:puny knowledge).
The terminal prints several ".... - is a directory" and ".... - no such file or directory".
Which logs shall i paste?

Why i cling to this configuration
30 - 40 gigabytes of applications + customization. You'll understand that reinstalling and reconfiguring that from scratch will take time. i have time, but not the patience. i don't want to do it all over again!
i have a fresh liveDVD and can access my files. It's at my fingertips! Can't i just overwrite what's broken? Or what do i have to save before i do a fresh install?


Why should this drive me away from Sabayon.... there is no failproof system. And Sabayon is fun to use. Install a proper backup solution, save an image of your install regularly, keep a backup of your files. No matter what system; there's always something silly that can fail. It could be stolen or hacked or fall down. Maybe you get mad and throw it against the wall or "someone" "accidently" deletes everything, just to regret it.

Hint to devs
To-do: automatic backup solution: when the installer recognizes a (large enough) free medium, ask whether to setup automatic backup (copying an image of the installed system). (Prompt to) overwrite existing backup when not enough space.

Re: Something's broken - Repairing the system

Posted: Mon Aug 23, 2010 23:48
by DontPanic
I applaud your initiative at implementing a custom boot splash scheme and qemu-style virtualization. Half the fun of linux is being able to play with such things.

Given your appetite for exploring new capabilities for you linux system, I strongly encourage you to take this opportunity to try to identify/fix the problem and learn more about how linux works in the process.

You've gotten off to a good start. You've found out how to chroot into your system from the LiveDVD, and you've demonstrated that your system is in good enough shape so that entropy still works.

I'm suspicious that some of your issues with sabutil are due to how your chroot session is set up.

You probably want to mount proc on your chroot session. From the LiveDVD session, run:

Code: Select all

mount -t proc proc /mnt/lvm/proc/
Also, when you first start your chroot-ed session, you should initialize it with:

Code: Select all

env-update && source /etc/profile
See if these suggestions clear up the output of sabutil, then maybe you can make an assessment of how much damage was done and why.

Re: Something's broken - Repairing the system

Posted: Tue Aug 31, 2010 21:32
by amoebios
Sorry, i was unclear;
i used sabutils from the INSTALLED (damaged) system AFTER running the LiveCD and rebooting.

Should i do it from the LiveCD? i just fear the big update before installing sabutils. Or is it installed by default?
Do i have to update everything?

PS: i sound much more knowledgeable than i really am. i have a lot of questionmarks floating around all those fancy words. Where does the output of sabutils go? And what exactly to do with it? :oops:

Edit:
Nooo, it's even more confusing: i wrote that post like a guide for beginners. i didn't actually install sabutils there (i had it already installed). "equo update" gave me errors (which i don't remember right now).

But that's about it. To sum it up:
1. system broke,
2. booted into live-system to save important files and restore passwords,
3. booted into damaged system to use sabutils,
4. posted this on forum.

Uff, that's roughly it, now. i'm sorry for the confusion, but that's how the inside of my head looks like.

Re: Something's broken - Repairing the system

Posted: Wed Sep 01, 2010 4:41
by DontPanic
For sabutil to give you any helpful information, you'll want to run it from within a chroot on your hard disk partition (after booting from the LiveCD/DVD/USB, and setting up a chroot to the hard disk installation).

The sabutil script is just a short list of commands that dumps out information that we have found to be useful in the past.

Re: Something's broken - Repairing the system

Posted: Wed Sep 01, 2010 16:33
by amoebios
Yay! i figured out how to use sabutils. There's a web address at the end of the output that you can write down.
Here's my log files:

Xorg.0.log: http://www.sabayon.org/pastie/4679
.xsession-errors: http://pastebin.com/7u3KSy7G (manually copied/pasted last part to pastebin)
dmesg: http://www.sabayon.org/pastie/4680
messages: http://www.sabayon.org/pastie/4681
rc.log: http://www.sabayon.org/pastie/4682
grub.conf: http://www.sabayon.org/pastie/4683
fdisk -l: http://www.sabayon.org/pastie/4684
kdm.log: http://www.sabayon.org/pastie/4685
Hardware Info: http://www.sabayon.org/pastie/4686


Have fun! :twisted: :wink:

Re: Something's broken - Repairing the system

Posted: Sat Sep 04, 2010 14:50
by amoebios
Well, i had some fun myself, did some libtest (don't remember the command, was recommended in chat) and a "equo --upgrade --ask" (entropy seemed to be working), but still where i left off.

Re: Something's broken - Repairing the system [Solved]

Posted: Sun Sep 05, 2010 14:50
by amoebios
Solved by re-installing.