Package Management System Time Machine AKA I have a Dream

[fenius]

This dream is about opensource and free software using user-friendliness and k.i.s.s. philosophy to overcome proprietary software in any field.

Do you know the OSX Time Machine thing? Well forget about it

What this is all about is simplifying linux package management process.

Say you world update in unstable branch because you like being bleeding edge but you go too far for your knowledge of the system. Say you update some critical package to test some new stuff and something goes wrong. Say, while updating, you simply forget to pay attention to some .conf. Sometimes there are a lot of them and you can easily miss to check one of the important ones. So things start going wrong and you have to start looking for a solution to restore the system to its original working situation... So you spend your time trying to fix things instead of testing new stuff as you wished to.

(This is a common problem to every distribution.
You can only avoid it staying in stable branch, sticking with well tested, stable but old software ...conservative/boring attitude here, where time seems freezed in some distros. Obviously this is just my personal point of view.)

The only thing you would want now is to get back, step by step, to the previous situation to restore the operating system to its working state, just as before (wrong?) updates took place.
That could be annoying ...do you have a time machine with you?

I was wandering if it would be possible to implement a package management system (Entropy? Portage?) with new superpowers:

...the one I was thinking about is the ability to recall a world update history, through a logging system that tracks down all changes that take place in the world file by installing and removing packages, so you can turn back to previous steps and fix things easier and faster in just a few commands/clicks/paces (reinstalling older but working versions of packages without worrying about remembering exactly what, when and where packages and .conf file where installed/updated).

You could restore the operating system back to the last working world file situation reinstalling old packages, libraries and .conf files.

This way the average user could rapidly restore the system if something accidentally brakes and focus his efforts on his original goal.

Don't you think it would be usefull to have the possibility of automagically going back and forth, step by step, in the process of updating/upgrading a linux system?

What do you think about it? Is there something similar somewhere? Do you think it's doable?

[Fitzcarraldo]

You mean something like System Restore in Windows? The only thing I have heard of for Linux that looks similar is TimeVault, but have not tried it myself.

[fenius]

Well, actually what I have in mind is something a bit different... it might be considered complementary to timevault ...and included as part of the package manager system.

Yes, it is to restore the system, but only the package manager update/install side ...not a backup utility (no diskspace needed, no user files saved anywhere), but a way to reinstall stuff with its original configuration files from online repositories.

Forget about windows system restore.
It is something useful only to distros with a package manager (linux/bsd way) ...to simplify the process for the average user.

I had this idea beacuse I often install the core system for friends and then build it as they prefer, trying to teach them howto to work with it, but ironically the package manager way of doing things often scares them away from linux.

Sometimes they go for an update without taking notes of every package they're upgrading and forgetting about .conf files.
So i don't know exactly what happened and I have to restore their system manually, trying to go back, step by step, to the point where everything worked and then reinstall new software the right way... this can be very time spending and often newbies get scared since they can't fully understand what i'm doing... they look at me as if I was a guru while this is not the case. It seems too much as if linux is just a niche thing and I'm some sort of illuminato, if you know what I mean.

I think that simplifying the package management process is one of the most important things we have to work on if we want linux to become more and more popular ...people get scared when they update the system and something doesn't work anymore. They don't know where to start looking at for fixing things. I think that a simple "get back to the previous working situation" option would be very appreciated by many and will help spread linux worldwide.

[sjieke]

I think this would be a great idea, but there are some issues popping in my head.

First determining the 'previous working situation' would be very hard, maybe impossible.
Lets say I world update my system and the situation before the update is choosen as 'the previous working situation'. The update broke a part of my system (lets say the sound), but I didn't notice it because all I did was quickly update my system and checking my mails before leaving on a trip. A few days later I come back, boot my system fire up the package manager and do a world update. The 'previous working state' will now be the one before my last update, so with no sound. After the update I decide to play some music while checking my mails. So I turn on my speakers and start some mp3 files, but... no sound. Ok, no problem I think, I will just revert to the 'previous working situation'. After doing so, still no sound... So I'm back to manually resolving the problem.

So what I would propose is instead trying to keep a 'previous working state' the package manager should just keep track of a history of changes, including the config files. So if you update some packages or do a world update, it somewhere keeps a history list as follows:

DateTime ----------------- Package --------- Orig Version ---- New Version
01/09/2009 12:01:52 --- PackageX -------- 1.1 -------------- 1.2
02/09/2009 17:12:23 --- PackageY -------- 5.6.3 ------------ 5.7.2

And the same for the config files, storing a copy of the original version or maybe the diff would be sufficient.
Then the package manager should have an option to show the history and also to undo a certain change. This should then automatically undo related changes (dependencies updated because of this change).
You could then also have an option to undo all changes up till a user selected one, etc.

This is a little bit different than backup up system folders, because undoing a change is just reinstalling the original version.
Space requirements would be minimal. Only a table (or a few tables) in a database to store the history in and some copies of the original config files.
So the amount of space required is determined by the size of the history. You could say for example all changes of the last 100 days should be kept, or a maximum of 1000 history entries, or... make the size of the history adjustable by the user.

[fenius]

Yes sjieke, that's exactly what i was trying to say ...I see you get the point and explained it better

[DHalens]

I'll write my opinion about the viability of this using entropy.
First, take a look to /var/log/entropy/equo.log Everything related to [un]installing programs is there. You may also search for "Protecting config file" to see which config files where not overwritten by default.
If I remember correctly, entropy overwrite config files (mostly but not all of them) if they were not changed by the user since they were installed by checking the timestamps. If before overwrite entropy checks if the previous config file was edited by the user and backup it if necessary, it may be possible to do it.

I see three issues so far:
- If you broke, lets say, the module for read/write the ext file system (is that even possible?!), there's no way entropy could fix it as it won't be able to read its own database to check which packages exist.
- A few packages can't be downgraded or removed. In real life, to get those packages causing issues is as hard as convince a wise old user to redirect /dev/zero to /dev/sda but who knows...
- Some packages use scripts before and/or after being installed (example: newer kernels removes extents option for ext4 drives from /etc/fstab). This is almost impossible to solve unless package maintainers do something for it (maybe add a little database with scripts to undo changes from the newer packages?).

[sjieke]

I'll write my opinion about the viability of this using entropy.
First, take a look to /var/log/entropy/equo.log Everything related to [un]installing programs is there.

I didn't have time to look at the file yet, but if I understand correctly all needed information is already there. So if you would manually parse that log file you probably could create the history yourself and undo changes by installing the previous versions of possible broken packages yourself. If that is the case then providing the core funtionality to undo changes in sulfur or equo shouldn't be to hard, either the log file should be parsed, or the info written in the log file should come in a database to ease the lookup.

But there are always special cases and possible side affects

Some of those can be solved by specifying some requirements/constraints.
One such constraint I can think of is that if you select a point in the history to undo. All changes past this point should also be reverted, to make sure everything is consistent. Because undoing one change from the past could (not nesscessary will) break other packages installed afterwards due to dependencies. (If needed I can provide an example here...)

Others will need to be solved by some work arounds:

If you broke, lets say, the module for read/write the ext file system (is that even possible?!), there's no way entropy could fix it as it won't be able to read its own database to check which packages exist.

Sometimes stuff will be broken with an unbootable or unusable system as a result. But shouldn't you be able to boot a liveDVD, chroot into your installed environment and undo changes there using the history functionality of the package manager to fix it? Maybe the package management UI (sulfur) could have an option to chroot into another environment, reading the database from that environment and thus providing the history/undo functionality there?

A few packages can't be downgraded or removed. In real life, to get those packages causing issues is as hard as convince a wise old user to redirect /dev/zero to /dev/sda but who knows...

If such packages get broken I guess every OS would be beyond repair...

Some packages use scripts before and/or after being installed (example: newer kernels removes extents option for ext4 drives from /etc/fstab). This is almost impossible to solve unless package maintainers do something for it (maybe add a little database with scripts to undo changes from the newer packages?).

I also think this is only solvable by giving the package maintainers extra work. Either they should provide undo scripts or they should backup the files changed by the scripts so that the undo could restore those files. The question is how many of the packages fall into this category. If it is let say only a small fraction, then this is doable, but if half of the packages fall into this category this would be a lot of work... The package maintainers probably have a better view on this and maybe even a possible solution...