[sandbox] Brainstorm: N900 - 'Save Me' - Disaster Recovery tool

[noobmonkey]

Good day everyone! - whilst i wait for Brainstorm to stop giving me network errors i thought i'd write this out.

Brainstorm link Here (http://maemo.org/community/brainstorm/view/brainstorm-n900--save_me--disaster_recovery_tool)

The inital idea with thanks to benny1967 (http://talk.maemo.org/member.php?u=2115) and ColonelKilkenny (http://talk.maemo.org/member.php?u=23712)

The general aim of this brainstorming idea is to create/develop a 'Save Me' function for people who have run out of rootfs space on their N900, it could possibly have other functions too.

Now, i am very aware that people should not delve into the development or testing repositories without the sexy official warnings - but as they are available, they will get used. That means, as much as we warn people, and they know it is their responsibility..... They are still going to do it, and the questions for support will end up in these forums.

So the Brainstorm proposal, is for a 'Save me' tool/application (Or even a choice in the bootloader application! - possibly)

The save me tool should be able to, for now:

Recover space in the rootfs area
Be simple to understand for most users


I agree extra functionality to fix other problems could be added, but for now i would suggest the basic function is to save a user by removing apps/free up space.

Some suggestions are as follows:

Remove all extras-development applications
Remove selected development applications
Remove all unused packages (A bit like a windows file cleanuip wizard i suppose!)
Throw out some output to the user showing current status - therefore allowing feedback to this forum/clever people, if there are still problems.

(Possibly forcing a reboot after the un-installs etc)

I would be very interested to hear any possible other functions the saveme tool could have! - as well as any better suggestions above if needed?

Edit : Still trying to link and add solutions - its sooooooooo slow!

[Rob1n]

You can free up a good chunk of space just by disabling the -devel and -testing repositories (which removes the cached app info for them).

[GameboyRMH]

I've heard that deselecting the extras repos and running apt-get update (edit: and apt-get clean and apt-get autoclean) can free up a lot of space...maybe it should offer to do things in stages, starting with this, then removing unused packages, then doing more invasive things like removing apps until enough space is freed up (35MB IIRC).

[chainreaction]

The problem with this idea is that the only use for this is for people who want the simplest solution possible. E.g. "This is broken, plz fix!". Therefore if this would be an application downloadable from the repositories I'd think a lot of people would download it and find later that their "apps are broken, plz halp". If someone comes here to ask for help, then giving information how and why to use the application would be handy.

[VRe]

What I have noticed that when the space goes full, I cannot use the application manager to remove anything. The dpkg from commandline also fails as there is no space. I solved this problem simply by:

# cd /var/lib
# mv dpkg /home/var_lib_dpkg
# ln -s /home/var_lib_dpkg dpkg

Now dpkg will always work and it is not necessary to clear space to get it working. Saved my a** few times :)

[noobmonkey]

I'm having a nightmare with brainstorm - i can't add the solutions i want and it keeps hanging - have i done it correctly?!?! aghhhhhh (At work - so i was hoping it would be quick!!!)

[noobmonkey]

Ok, ignore previous post - solutions are slowly being added - anyone fancy brainstorming the brainstorm system and adding a "Add multiple solutions" button? (Or at least make it visible!) hehe

[wheatbisc]

The problem with this idea is that the only use for this is for people who want the simplest solution possible. E.g. "This is broken, plz fix!". Therefore if this would be an application downloadable from the repositories I'd think a lot of people would download it and find later that their "apps are broken, plz halp". If someone comes here to ask for help, then giving information how and why to use the application would be handy.

And that problem is exactly the winning factor in this idea. We all want to make the Maemo community grow, and for that we want different kinds of people to join in, instead of only the high-tech, "early adapters". Simplicity, we need it!

I think this is a great idea, and also for the lazy ;) No moar spays, use "Save me"! ;)

[noobmonkey]

And that problem is exactly the winning factor in this idea. We all want to make the Maemo community grow, and for that we want different kinds of people to join in, instead of only the high-tech, "early adapters". Simplicity, we need it!

I think this is a great idea, and also for the lazy ;) No moar spays, use "Save me"! ;)

Thankyou wheatbisc i think you hit the nail on the head so to speak :)

People like options - and yes this application may not solve world poverty or what comes out of Gordon Browns mouth, It should give a helpful tool to some.

It could easily have a selection of one click fixes as well, but for now i'm just brainstorming the initial use :)

[chemist]

Something like deborphan will help too.
Clear tempfiles
Clear caches
Clear /var/log/ (and reset debug modes,maybe a watchdog for oversize logs will help)

[noobmonkey]

Something like deborphan will help too.
Clear tempfiles
Clear caches
Clear /var/log/ (and reset debug modes,maybe a watchdog for oversize logs will help)


Not heard off it - i'm assuming this is a debian app? - if there is already something that does this it should really be promoted and be given a user friendly name?

[PowerUser]

Crash recovery without data loss (or with limited data loss) is generally not an easy task and requires working brain. Look, you need to:
1) Fetch failed partition from device.. Haha, now if device already failed to boot, you simply can't do it at all due to quite stupid boot loader. At least I know no documented ways of doing so.
2)Then you need to mount filesystem image on your PC from a file. No, Windows can't do that at all (especially for UbiFS, etc). So you would have some troubles with getting a single-button windoze app to fix this. Surprise, surprise.
3) Then you need to edit partition to fix issues. It's not very easy to decide automatically which "useless" data to remove.
4) Then corrected partition have to be uploaded back to device.

Do you really think all this is easy to implement as single-button solution and worth of required amount of efforts?

For relatively simple methods, use flasher to upload empty FS image to a device, this may be counted as "freeing disk space" too, though it's destructive and all installed apps will be gone of course :P.

P.S. IMHO it would be cool if Nokia would improve or replace boot loader + flasher bundle to allow us more advanced recovery like fetching problematic partition from device, fixing it manually and re-uploading it. This could allow to recover user's data in the case of emergency, or even fix booting, etc. Some decent boot loaders like u-boot allow this trick. Nokia's ones are not documented and do not allow such things at all, at least I found no obvious ways to do so, etc. In this sense Nokia devices are bad in terms of "debricking". I seen devices like routers, NASes, etc with much better and smarter boot loaders who allow much better crash recovery procedures.

[ruskie]

Provide an initramfs with a 1-2 second delay to type in something like e to get a shell... requires a kernel with fbcon. Then one can operate on the device and fix it.

Would be nice to get uboot instead of NOLO as well.

[ndi]

I would frankly love a tradeoff app.

E.g, locate the largest stuff that is on the rootfs, then move and link it, ideally with checkboxes.

I, for one, need dome libqt4 for an app I can't give up. The whole package consists of one libqt4-whatever.so that's 20 MB. Add a checkbox, move to installable fs and link back.

Unchecking the box in front of it deletes the link and moves the file back so that apt-get can do stuff to it. For example uncheck apps before updating.

It's not hard, the app Storage Usage builds a list of packages, ordered by size, and greens or reds them as they are optified or not. Now add an option to push the package back and forth and we're done.

Lovely part is we could push packages that come with the OS (within limits), such as gaining a meg or so via the welcome video. There's also a few megs in Chinese fonts. If anything ever displays Chinese on my phone then correctly rendered is perfectly equal to incorrectly rendered because I can't tell.

That would be useful. Uninstalling stuff I can do by hand via point and click. Also, uninstalling all alpha stuff is pointless as extras have non-fully-optified packages, or are themselves non-fully-optified.

[Rob1n]

If you've installed the firmware update, you should be able to gain space by uninstalling & reinstalling libqt4 - new installs will use a higher compression setting with the new firmware.

[noobmonkey]

Crash recovery without data loss (or with limited data loss) is generally not an easy task and requires working brain. Look, you need to:
1) Fetch failed partition from device.. Haha, now if device already failed to boot, you simply can't do it at all due to quite stupid boot loader. At least I know no documented ways of doing so.
2)Then you need to mount filesystem image on your PC from a file. No, Windows can't do that at all (especially for UbiFS, etc). So you would have some troubles with getting a single-button windoze app to fix this. Surprise, surprise.
3) Then you need to edit partition to fix issues. It's not very easy to decide automatically which "useless" data to remove.
4) Then corrected partition have to be uploaded back to device.

Do you really think all this is easy to implement as single-button solution and worth of required amount of efforts?

For relatively simple methods, use flasher to upload empty FS image to a device, this may be counted as "freeing disk space" too, though it's destructive and all installed apps will be gone of course :P.

P.S. IMHO it would be cool if Nokia would improve or replace boot loader + flasher bundle to allow us more advanced recovery like fetching problematic partition from device, fixing it manually and re-uploading it. This could allow to recover user's data in the case of emergency, or even fix booting, etc. Some decent boot loaders like u-boot allow this trick. Nokia's ones are not documented and do not allow such things at all, at least I found no obvious ways to do so, etc. In this sense Nokia devices are bad in terms of "debricking". I seen devices like routers, NASes, etc with much better and smarter boot loaders who allow much better crash recovery procedures.


some very good ideas in there :)

I think its clear in my aim that the audience is the non techy. This doesn't discount more advanced groups of tools for advanced users, but when we tell people, get this app to do this, then sudo that, apt that etc..... a small collection of useful tools could be good.... (For some, again not all)

In this case a simple SaveMe tool, its aim is to help people when the rootfs is full. - now that could be done in a number of ways - and hopefully people can add their solutions to the brainstorm :)

I think a few other tools could be made to almost make a basic survival toolkit :) - do some of the advanced commands, but with some safety (Yes i understand the irony and probable issues with that) :)

[chemist]

Not heard off it - i'm assuming this is a debian app? - if there is already something that does this it should really be promoted and be given a user friendly name?

yes deborphan is a debian tool to point out unused and undepended packages - orphaned .deb's


man deborphan gives
DESCRIPTION
deborphan finds packages that have no packages depending on them. The default operation is to search only within the libs and oldlibs sections to hunt down unused libraries.

[noobmonkey]

yes deborphan is a debian tool to point out unused and undepended packages - orphaned .deb's


man deborphan gives
DESCRIPTION
deborphan finds packages that have no packages depending on them. The default operation is to search only within the libs and oldlibs sections to hunt down unused libraries.

Sounds good :) - could it be used in this way, or as part of this idea in a simplified gui for people? (Or is it possible?)

[chemist]

Sounds good :) - could it be used in this way, or as part of this idea in a simplified gui for people? (Or is it possible?)

if there arent any conflicts... we could just ask the HAM guys to include it as part of it. (file a bug?)

deborphan drops a list of libs which could easily be used to do something like

apt-get remove <deborphan-list>

but I guess its not a very good idea to do such things blindly, is it?

[Stskeeps]

Two ways to go about this, - a specially crafted rootfs on microsd that is a rescue mode of sorts and an accompanying kernel that boots this rootfs. Or one manually installed which uses fanoush'es bootmenu.

[MrGrim]

Here's an idea: we could make an app called safetynet. It will do nothing except create a considerably-sized file on / (the size is to be discussed, maybe 1 MB should do). It will be highly recommended to anyone working with -testing/-devel, or who just want to install every app in the repo :D
When the chocolate hits the spinning blades, our app (well, it does do something) will be called in to conveniently remove this file, thus giving the user just enough maneuver room to uninstall other apps

[egoshin]

Just link /usr/share (and ...) to somewhere outside of / FS and allows the installation in it only. That is a fundamental solution - split stuff to 3 stages: (1)boot-and-mount, (2)basic-GUI-start and (3)optional-and-manual-stuff-(system-can-run-w/out-it).

I know, it is a lot of work, but cant resist.

[chemist]

Please post solutions to the brainstorm item so people can vote on them once it leaves the sandbox.

[noobmonkey]

*Bump*

any other ideas - suggestions or people willing to look at the brainstorm and propose things? :)

[chemist]

Now it is in need of votes to leave sandbox status ;)

[RevdKathy]

I voted. (I thought I also posted). :confused:

I'd love to do a bit more by way of testing and feedback, but the thing stopping me is a fear of doing something to My Mo I can't recover. I haven't yet had to reflash (*quietly thanks all available deities*) and it's not something I'm in a hurry to try. This would be perfect for me.:)

Small suggestion: safe mode and roll back? - probably too big to implement. But something that would let you launch in safe mode and roll back to a previous point would be perfect. I could create a 'roll-back' point before installing anything I wasn't sure of, then just go back there.

(yeah, I have big ideas and no notion whether they'd do-able! ;p )

[noobmonkey]

I voted. (I thought I also posted). :confused:

I'd love to do a bit more by way of testing and feedback, but the thing stopping me is a fear of doing something to My Mo I can't recover. I haven't yet had to reflash (*quietly thanks all available deities*) and it's not something I'm in a hurry to try. This would be perfect for me.:)

Small suggestion: safe mode and roll back? - probably too big to implement. But something that would let you launch in safe mode and roll back to a previous point would be perfect. I could create a 'roll-back' point before installing anything I wasn't sure of, then just go back there.

(yeah, I have big ideas and no notion whether they'd do-able! ;p )

i do like the idea of a roll-back! (We are becoming more and more micro-nux'd or li-softed!!)

But good ideas - i feel what this could turn into is a start out safety button - but possibly a suite of mini tools, like roll back, etc etc....

Still need some lovely developers :)

[benny1967]

Crash recovery without data loss (or with limited data loss) is generally not an easy task and requires working brain. Look, you need to:
1) Fetch failed partition from device.. Haha, now if device already failed to boot, you simply can't do it at all due to quite stupid boot loader. At least I know no documented ways of doing so.
2)Then you need to mount filesystem image on your PC from a file. No, Windows can't do that at all (especially for UbiFS, etc). So you would have some troubles with getting a single-button windoze app to fix this. Surprise, surprise.
3) Then you need to edit partition to fix issues. It's not very easy to decide automatically which "useless" data to remove.
4) Then corrected partition have to be uploaded back to device.

Remember that the original use case was a lot simpler:

There was a cry for help from a user who couldn't perform the SSU because his rootfs was full. He was told the usual strategies to free some space, but it was evident he didn't feel confident enough...

What we thought would be nice in such a situation would be to reset the device to "a sane recommended state" for such a non-techy end user. Which means: Get rid of applications installed from repositories other than Extras, Ovi and the other 2 Nokia ones. Disable said repositories.

There will be a lot of data loss, because you simply uninstall or downgrade an unknown number of applications. But then it's meant to be the next best thing to re-flashing for such users. Also, it server the purpose of bringing the user back to where he "belongs": to Extras. Away from -testing and -devel.

[qgil]

Do you mind editing the title of the brainstorm?

"Brainstorm: N900 - 'Save Me' - Disaster Recovery tool"

It indicates nothing about the real problem trying to be solved: a tool solving the problem of having run out of rootfs space.

Thanks!