Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Hello, I have a question for the experts
I have a N900 with the broken micro USB problem, so no flashing possible, now my question If someone make a backup of a new N900 can I use that file to flash my N900 using backup menu?

Thank you
Edaz

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I got a Error during Backing up RootFS:
Have I to worry? :D

EDIT:

Oh, and I have a other Bug to Report:
There is no difference between
o) USB Mass Storage Mode(read-only).
and
w) USB Mass Storage Mode(read-write).

I was hat booth able to delete files from the SDHC Memory Card. Only the internal Memory was read-only.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

So, small update and after more than 2 hours of testing. :D After Checking the created Files I was brave enought to flash my device with vanilla rootfs and eMMC image.

I reinstalled rootsh and BackupMenu and restored after this the root and OptFS Backup I created before.

As far as I able to see everything worked just nice! :)

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Mine's broken.

I did a backup and a restore and rootfs checksums never match.

And with this communication error popping out from nowhere it's really, really annoying. Any help?

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Hrmm. That's odd. Every time I got that it was either after installin or removing stuff with FAM. And now, after a reflash, FAM complains that some of the apps are not installable. Which is odd because some of this stuff is old. Like Numpty Physics and Extra Decoder Support.

HAM installs them just fine.

Could this be a run-amok FAM? That easy? Investigating.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

NDI, I think you are using the old, 0.3x version. You should upgrade to the latest nightly of -this- version(just download and extract the .tar.gz), which will also overwrite the old version.
This version has no specific "checksum" function - It's not easily possible. Unlike the old version, this one works with files, so each file would need to be checksummed and compared - a non-trivial task.
It -should- work right though, as the UbiFS filesystem is error-correcting, and so should prevent disk errors etc.
The old version was -not- aware of bad blocks, and so could easily write into a bad-block, creating corrupted data randomly(specific to each N900). This is probably the problem with your system; you ended up with a bad block in some specific file.

Helmuth: I'll look at the read-only/read-write; I'm not sure entirely how to make it Do The Right Thing(TM).

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Oh good, because I eventually run into the same error no matter what. I noticed you announced a "new version" but I assumed v32 was it.

That explains a lot, including why my N900 behaves like it has a bad block.

How do I surface-check it? Every reflash it seems to forget the bad blocks. Can I fsck it? Fill it with random data and the delete the file? Would cat /dev/random > /fillmeup work?

I'm new to Linux. In Windows I can use this method or writing and reading the disk to force a surface scan. OTOH, in windows I need to add -R to chkdsk to do a surface scan.

I'm afraid that if I reflash again and install again I hit the snag again. OTOH, with BackupMenu working (assuming it will) I should be able to revert to a semi-installed and configured OS instead of zero.

Edit: I installed as per first post indications, seems to work and it claims to use tar instead of dd, so I guess I'm right. Read the thread back, so you know fsck fails. As is my case, fsck is quite important to me.

I did a backup am restoring as we speak. Will report back.

Oh, wait, it's done.

Great, it booted up. so it works. Thanks.

Oh, and, two more things.

a) why are suggestions as they are?
b) there's a typo just before restoring (possibly before backup as well). The menu spells "procede", should be "proceed".

Well, back to work. At least I can restore and then try to see what breaks it. At least I can regain a functional phone on the road. Thus far, I needed a PC and the Nokia cable. And a virtual XP.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

One, that's the problem with raw NAND, like our RootFS is on - It doesn't keep track of bad blocks. That's all up to the OS and filesystem.
UbiFS should automatically mark all blocks bad as they are found, but if you overwrite it with an older version of the FS(like happened with the 0.3x BackupMenu), bad blocks could be marked good.

With the SD card, there is a chip which handles bad blocks; No bad blocks are ever shown to the OS or filesystem provided things are working right, so doing a surface scan won't help anything.
This means that writing a raw image is perfectly fine; it won't hurt anything as no bad blocks are ever read or written.

This is why you are supposed to be testing/using the new version, and -not- the 0.3x series currently uploaded.

FSCK should work(better) in this version, though -IIRC- I don't think there's a "real" check module for UbiFS - it just checks for file coherency.

Restoring should be possible through the restore functionality of the current version, or through making an image and restoring that.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Robbie, this time I could backup at least without errors afterwards when I reboot. Thanks :)

SDHC ext2/ext3 formattetd
 

Hi,
i tried to make a fullbackup (rootfs and optfs) to the internal SDHC card, with is ext2 formatted and full functional, when running the N900 in normal mode.
The BackupMenu ends up with Backup successfull, but nothing was backed up. On a vfat formatted sdhc card, the backup works. Seems, that the ext2-card is not mounted by BackupMenu.
How can I fix this problem?

I tried to check the mounting issue within the build in terminal, but i need a short description, how to use this terminal (esp. how to mount, cd). The only command with success is "ls" with or without parameter. All other commands returns "press backspace", but this returns back to #.

Thanks.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I (ab)used what I knew of filesystems to force a check. Once freshly flashed (and known as good), I used the CLI to copy a known correct file form the SD to the rootfs several times, until the rootfs was filled. Then read back the files (copied to nul). This should have triggered any bad block on the surface to be marked as bad.

Once this was done, I installed diff and did it again, copied file from the card to rootfs in several instances, then used diff to make sure it's all good, and finally removed.

It all went well, no errors, so I'm going to assume that either this wasn't a bad block issue (though it certainly seems so) or it was correctly marked as such. So far, so good, I have most of my must-haves installed and no glitch.

Any way to know if I have bad blocks?

Also, fsck seems to not work in the latest version? Maybe it does, but my version spits some errors and fills the screen with what seems like UINT32 values in bright green. I'm afraid to keep trying.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Hm... I'd probably better add a way to copy the fsckstatus file to your eMMC in case you want to look at it... I assume it's working -correctly-, but I don't know what the output means from your description.

Also, as I said before, you do have bad blocks. It's inevitable. The -filesystem- marks them as bad; this means that if you re-flash, the new FS has to re-mark them. Also, copying a file to it will - at the FS level - automatically mark all bad blocks as bad and work around them. Thus, you won't see any problems(aside from a momentary "pause" as it re-writes the content to a different block).

However, as I've been trying to tell you, if you use the old 0.3x version BAD BLOCKS ARE INCLUDED IN YOUR BACKUP! They then get written over the current filesystem, IGNORING ANY (new) BAD BLOCKS IT FINDS.

If you want more information, go talk to Stskeeps on the IRC. He knows a lot about the raw nand flash and can explain it to you.

Solution: USE THIS VERSION OF BACKUPMENU. Not the one available in the repositories.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I'll test.

So reflashing doesn't do anything for bad blocks? I'm thinking the controller for the NAND should, after all, if the block is under the system image it's bricked. I'll take your advice and bother Stskeeps.

Anyway, after further investigation is seems I may have the cause of the errors. I don't want to throw stones just yet, I want to do several tests before I make a firm affirmation, but it seems it wasn't bad blocks that were the root of my problem, but a file system problem.

I am. And, I have made a backup and restored it successfully. Additionally, I have again triggered the error in my phone and BackupMenu v43 has successfully repaired the problem.

I am doing almost hourly backups as I diagnose the issue, so I'm giving the script a thorough check. :)

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

That's the whole point - there -is- no controller. This is what screwed me up with my previous version - I'm used to SD cards and HDs where you can go ahead and copy the entire partition or disk using dd, and everything works fine. In this case, I learn(way after the fact) that the OneNAND used on the N900(part of the CPU/SoC, btw) has no controller, and as such requires a special filesystem to handle it - This is the reason for Jffs2 and Ubifs.
When you write an image, it has to be a special "ubinized" version of the UbiFS. I'm betting that between that and the flasher utility, it works around bad-blocks, at least on the first write.
This would explain why, for some people, BackupMenu v.3x rootfs images would flash via the flasher, but not work if written back via dd.


Huh - Not sure how that'd work. My system's supposed to copy each file using "tar -cfp destination.tar /inputfolder" - There's only two "active" lines in the entire backup section!
I'd check that you don't have a missing or "corrupt" file in your input - When I used my own old version, I ended up with a couple of corrupt files, which weren't able to be read when backing it up with the new version, so they were missing in all subsequent backups. Fortunately, in my case, they were unused files(other languages).

Ah, good. I was worried about you using the old version because you were talking about checksums, which aren't used in the new version.

Also(just curious), did you restore your rootfs via BackupMenu, or did you make an image and flash it?

And, of course, feel free to edit/fix it - It's GPL after all. I would like to see any changes you think would improve it, however - I'd like some outside additions.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Robbie, the file in http://pastebin.org/287361 needed for producing a flashable rootfs isn't available any more. Would you please attach it in the forum or elsewhere for us to get? Thanks

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Is this V2 you're talking about or the old version? Both SSH and USB networking worked on the older version and has saved me from doing a restore (from an out of date backup) on more than one occasion. If it doesn't work on V2, please, please get it working :) since terminal mode doesn't really work being able to use SSH is a lifesaver.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

@debernardis:
Fixed the missing file.

@kureyon:
Huh, that's cool then. I'll see what I can do...

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

@debernardis:
Fixed the missing file.

@kureyon:
Huh, that's cool then. I'll see what I can do...

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Thanks Robbie, but now there is another issue.
Since I had some problems in upgrading to pr1.3, I restored my backup done with your version 0.47. Everything went fine in restoring, but in the end I had some 10 megs free less in rootfs, i.e. about 27 megs. I reminded it was much more!
Thankfully, I had also a backup of the same rootfs done with your version 0.32 - the one saving in img format. So I restored with the old version, after having turned to the old backupmenu. In the end, I had 37 megs free.

How comes? Is it possible that untarring makes everything bigger?

Thanks for your patience ;)

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Did you reboot after checking? It might be the reclaim thing on UBI - IIRC it only releases or repacks or whatever in a reboot. The reboot just after restore wouldn't have cut it, because it's done on shutdown, not on power up.

Anyway, I take it it works fine in 1.3. Good to know.

Also, RobbieThe1st, when I said I may have isolated the problem in software I didn't mean yours. It seems that my troubles almost always begin when installing a package, or rather when a package I install pulls some dependencies.

E.g., installing Alarmed pulls several Python libs and in my last attempt doing that crashed it. I had Alarmed installed a long while with no issues, but at some point it went crazy because of an issue in Python - the list of alarms didn't update unless I restarted the app.

I've had problems ever since I started using Alarmed with the optified Python. I'm guessing that it nukes the cron or whatever it's called and some service fails to start.

Bootup takes way longer than usual and the error was this. Also, randomly, email takes huge gaps in updating. For example it sometimes doesn't update for hours until I start the email app then it resumes as normal.

Perhaps installing it and it finding some jobs in /opt convinces the application to re-insert the corrupt jobs in crond's list.

Still digging. But first, PR 1.3.

ETA: Doesn't work in 1.3 as far as I can tell. After "Mb reaches 254.8 it stops advancing. Time is around 30 or so. After that, time advances but no end in sight. By second 300, the Size: becomes empty, time still counts. No power button or keyboard controls. Needs a battery pull AFAICT.

ETA2: Reinstalling it seems to have fixed the issue.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

So pre PR 1.3 upgrade, I successfully uninstalled NitDroid, multiboot, and the previous iterations of BackupMenu.item. I successfully downloaded, installed and ran the new version of BackupMenu, creating a backup of both rootfs and optfs.

I then made some changes to my MfE relationship, deleted a game that was taking up too much application memory (OpenArena), and updated a few apps. I then wanted to re-run BackupMenu and create a new backup. However, the bootmenu screen would not show up (I've now tried three times) after rebooting with the keyboard open, and my N900 just boots up into Maemo.

I then reinstalled everything according to post #1 and still no luck. Any ideas?

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Make sure bootmenu-n900's installed and working?
For all I know, bootmenu-n900 doesn't work well with PR1.3, and there's nothing I can do about it.

@ndi:
I'll add a "break" from that loop based on a key - say, X or something....

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I haven't yet upgraded to 1.3 (the OTA still hasn't shown up for the UK), but FSCK isn't working for me either. Well, it works for rootfs, but not for optfs - comes up with an error about /etc/fstab and then I can't read the rest as it's garbled..

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Ok, new version all: http://robbiethe1st.afraid.org/Backu...0101026.tar.gz
I've:
1. enabled usb networking stuff/network terminal
2. improved the feedback for FSCK and added a dummy fstab for it. New feedback shouldn't extend beyond the sides of the active screen area. However, I haven't yet gotten a solution for if there's more lines than can fit on a screen, so it'll run off the bottom of the screen sometimes.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I tried posting maybe 20 times. TMO outages make this very difficult.

The latest version you posted doesn't work for me. I used the first post as guide, used tar to unpack to root (paths seem OK), then used the copy command for the menu item, them the two chmod commands.

When booting up the menu works, it lets me go through but when it is time to actually back up it does nothing. "Backup complete" comes up in under a second. It's like tar never ran.

What did I miss?

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I second maybe third or fourth this. :(

Any way I can debug?

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Fixed: http://robbiethe1st.afraid.org/Backu...01026-2.tar.gz
I didn't understand the correct way to get the result code of a function, so - evidently - it just skipped over that line.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Aaaaaaargh. Bricked again. Damn, Damn, Damn.

Restore bricked my phone, again. Don't know if it's the backup that died or the restore but frankly this is getting way over a little inconvenience. Alarmed breaks my telephony on install, and frankly I don't know if it's the application or the dependencies - it could die at any time and with restore function in Backupmenu dead I'm boned.

Doesn't boot any more after restoring. Which means, basically, that I may be reduced to manually reinstall everything. I'll have to go home, flash it again, install an older version and try an older backup.

And I'm at work, with no XP and no microUSB cable.

Sigh. I can't spend the day with no connectivity. I'm driving home.

Damn.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

This is why I bought extra USB-cable which I carry ALL THE TIME in my pocket and I have also set up virtual XP running in my work PC ;-)

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

My work PC is a quad core running W7 x64 with no hardware virtualization. I can't flash from work. And, no VMWare at work, either, no license.

My laptop has been seized my fiancee. Facebook.

ETA:

Oh, and, seeing as only one version works every few releases, wouldn't it be better to leave the older versions linked as well? Not to mention it would be mighty helpful if it were part of -devel with a postinstall script, that way I wouldn't have to manually go through the process again?

Also, your install instructions are missing a step, installing rootsh. Without it, the instructions fail with "Enable R&D mode if you want to break your device"

And finally, at least one version made an empty backup, and one (I think latest) made a suspiciously small backup.

I'm attempting to go with a backup from 4 days ago and the version from 22nd.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

http://www.virtualbox.org/wiki/Downloads
Edit: Yea, there -may- be problems with my software. That's what you guys are here for - This is -alpha- software, You guys are getting it after I've tested it somewhat, but I'm one person and not perfect.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Or: http://www.vmware.com/products/player/ (yes, it's freeware)

Or burn ubuntu live-cd...

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I'll try at work. Right now I'm at home, unable to restore.

Wait, it booted. So, latest version that actually restores for me is from 22nd, when I posted that it restored successfully.

Also, the image sizes are suspicious to say the least. Here we go:

As you can see, the size of my backups vary wildly. Last stable version, 25th, has my rootfs at 268M. ICYWW, the Pre13r is the pre-PR 1.3 root image, whereas the one dated 25th is post-1.3. Sizes are similar, and a gain of 6 MB through a reflash is understandable.

Now, on the 26th, you released version 20101026, and the images are 0 MB for root (dead) and 92M for opt, where it's clear that opt should be around (over) 750. Upgrading to 1.3 has no excuse to vary my backup that much.

On 27th, 26-2. Rootfs backup is 2M (!) and opt is 900M. After and unpack and diff, it seems the 200M was the result of my installing gnumeric, angrybirds and a few others, so it seem opt is fine, should be 900M. The diff shows that optfs only gained. Deleted files from Pre13o to the 900M image are a few cache files and some files I deleted myself.

20101027-backup-rootfs.tar contains
/etc
/dev
/bin (partial)
and an unexpected end of archive error.

20101027-backup-optfs.tar contains what is probably expeced of it.

20101026-backup-rootfs.tar contains nothing, obviously.

20101026-backup-optfs.tar contains /user and an unexpected end of archive

20101025-backup-rootfs.tar is valid, as we know, and could be restored.

So, basically, it's broken. Moreover, there is no indication of the backup failing, so a restore renders the device unbootable. Not unbootable like the boot menu works, but unbootable like the logo comes up but not the backlight, then power down. Needs reflash.

And to add insult to injury, my backup is 1.2. I have to do OTA every time to 1.3, then restore power kernel and then reinstall apps that were broken by OTA.

--

Suggestion box: Could you ask the user for a backup name? It's a lot useful than date, mainly because if I have a pre and a post install backup I can only guess. Sure I can rename manually, but if it breaks I'm lost. You could append date after the name, if needed. Is it too complicated to read from console?

Also suggestion box: Could you remember the result from the rootfs backup and display it at the end, with opt? Every time I initiate root+opt backup I leave the phone and after 15 minutes when I look at the screen I see "Rootfs backup complete" and it waits for me to press a key.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Yes, the 10201026 was broken - I already agreed to that and fixed it.
20101026-2... I dunno. I just did a test with my PR1.2, and it seems to be working alright... It gets to ~296mb, then finishes, so I assume it worked OK. I'd check to make doubly sure you have the latest.

Suggestion 1: I'll see. I did it this way because I didn't really have a way of getting an input name. I'll have to write a new module to do that... Possible, but'll take a bit

Suggestion 2: Yea, I'll have to do some extra coding though...

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Maybe 1.3 broke it. I now reverted back to 22 and restored back to 1.2 via restore. Now updating to 1.3. Once it's done, I'll try to make a backup again and a restore, see if 22 works under 1.3 - that should clear up some things for both of us.

I'm glad you think the suggestions are good. Oh, and, note to self and a caveat for everyone else:

Do not restore rootfs only if you upgraded to 1.3.

I assumed it would work since it's only the OS that's being updated, however, doing that broke several things. Among others, some paths have changed, some apps and widgets are missing images. Furthermore, some apps are broken, assumingly they moved paths and some libraries use different paths.

What's worse, HAM thinks you have 1.3 (doesn't offer to upgrade), but version says 10. (PR 1.2). It's in limbo and apt-get install --reinstall mp-fremantle-generic-pr doesn't fix it. It fixes the version, but not the applications.

Reinstalling applications also won't work, 1.3 has a different repository.

They also moved some files around.

So, no. If you don't have a root+opt backup, don't do it.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

I just backed my stuff up, and am upgrading to PR1.3 right now.

BTW, I found out that erasing all files from the Optfs(via a special feature of BackupMenu), then flashing PR1.3 does work, and makes a bootable system - Flashing the eMMC, therefor, should not be needed.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Update, version 22 does indeed back up and restore PR 1.3, if reinstalled after OTA.

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Help me understand what you just said please -
you reinstalled BUM after upgrading to PR 1.3 by means of OTA
or
you upgraded to PR 1.3 OTA but then reinstalled PR 1.3 ?

Just trying to be clear on what you meant,
thanks :)

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Installed 22
Backed up in PR 1.2
Killed the device
Restored to 1.2 >> success

Installed 22
Backed up in 1.2
Upgraded to 1.3
Restored to 1.2 >> success (see above warning in bold)

Installed 22
Upgraded to PR1.3
Installed 22 again (*)
Backed up 1.3 using 20101022
Killed the device again
Restored to 1.3.

That means, basically, that version 22 is solid whatever way you go, provided that you (*)reinstall it after OTA. I can't swear it won't work after OTA, but I did reinstall it to be on the safe side, so all my tests are with a reinstall after OTA.

(*)By reinstall I mean followed the instructions in first post. Since I already had backup menu and a working OS, I just went to the tar command to overwrite the files (and the the rest of the commands).

By now I have a BM.sh that does it, in /media/mmc1/Ndi/MB.sh. Along with other scripts I'm tired of running, along with a backup of repository definitions and a few other files. I just wished the flasher image came with rootsh installed. When recovering from a failed restore, I flash, followed by rootsh, followed by an "sh /media/mmc1/Ndi/BM.sh" and a restore.

I'm down to 2 minutes. :)

Re: [Testers needed] BackupMenu V2 - OS backup & restore
 

Lets see. I:
1. Backed up my system with V0.48.
2. Added a "special feature" that lets me delete the contents of the Optfs.
3. stuck that version on my N900
4. Booted into BackupMenu, erased the Optfs
5. Rebooted and flashed PR1.3(no eMMC flash)
6. Let it boot(It booted fine, worked great).
7. Opened the app manager, started restoring apps.
8. After apps were restored, used tar to extract my ApMeFo configuration from my PR1.2 backup and install it
9. Installed BackupMenu V0.48.
10. Rebooted, did a backup. The resulting files are within reason, so I assume it worked fine.
11. Copied the backup to my desktop incase something goes wrong.
12. Changed a few things, like the startup video.

13. Restored to my backup. Worked perfectly - Video was the default.

ndi, the latest worked fine on mine. Personally, I'm not sure how there could be a difference between the older one and the current, as the backup code is identical. :\