maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   MeeGo / Harmattan (https://talk.maemo.org/forumdisplay.php?f=45)
-   -   [HOW TO] Format and use a ext2 MyDocs partition. (https://talk.maemo.org/showthread.php?t=86662)

Alexislavie 2012-09-08 13:52

[HOW TO] Format and use a ext2 MyDocs partition.
 
3 Attachment(s)
This tutorial with the ext2Manager app is part of the Maemo Coding Competition 2012 ! (Desktop, System & Utilities + Beginners)

This is a complete guide on how to format and use your Nokia N9 with the storage memory formatted in ext2, this tutorial should works with any operating system, I tested N9 "Mass Storage" mode in ext2 on an Linux (Ubuntu) system and a Windows 7 system, I didn't test it with Mac OSx but it should works too.


Basic acknowledgement :
Difficulty of execution : Medium.
Time required : +02H00. (add +01H00 for Macintosh and Windows users).
Tested on : N9 PR1.3
Required level for understanding commands : Advanced (Linux users), Expert (Windows users).

Pros :
+4GB file support.
no file fragmentation.

Cons :
POSIX file permissions (I made a simple app to help facing this problem, so you should not care about it).

Resources :

ext2Manager
Ext2fsd v0.51
All other resources are available at the end of this tutorial.

Requirements :

- You need a computer with one of the following operating system. (Windows XP and after, Linux or Mac OSx, or anything UNIX based).
- You should know how to flash your device (http://talk.maemo.org/showthread.php?t=82693).
- Your Nokia N9 should be fully charged.
- Some knowledges about POSIX file permissions would be good for you to understand this tutorial, if you don't know about it look at this page : http://en.wikipedia.org/wiki/Filesystem_permissions.
- You need developer mode.
- You need to backup all personal data stored on your N9 (Media files, contacts, messages, apps, etc...).

F.A.Q. :

- Does every app (including pre-installed apps like Nokia Maps) still works after applying this tutorial ? - Yes, files read/write access remains the same as before.
- Does "Sync and connect" mode works too ? Yes, you can still sync your N9 with either Nokia Suite or Nokia Link.
- Does the SDK mode still works ? Yes, no problem here too.
- I do not understand what POSIX's rights are, can I follow this tutorial anyway ? - Yes, this isn't a problem.
- Is there a difference of performance between ext2 and fat32 ? Not really, follow this link for more details : TODO
- Will I be able to apply Nokia updates to my phone ? Yes, this hack is totally invisible, in regards of Nokia updates.
- Will this brick my warranty ? No, it shouldn't.
- Does the Nokia flasher works even if the MyDocs filesystem has been changed ? Yes, as this tutorial do not modify any partition size, the flasher do not detect any difference, and format the MyDocs partition back to fat32.
- Do I need inception or OpenKernel mode ? No, they aren't necessary at all.

Disclaimer : Please follow exactly the steps mentioned below, as this tutorial modify important system files, I claim not to be responsible if you brick your phone and/or lose data. I tested like five time this tutorial, and my N9 never bricked while executing it... Just read carefully the commands and nothing will go wrong.


Let's start !



STEP 1 - Installing an ext2 driver for Windows (and Macintosh)



Windows by default can't detect/read/write ext2 filesystems, to fix this we first need to install an ext2 driver on it, otherwise you won't be able to connect your N9 in "Mass storage" mode on your computer, after formatting it in ext2 (Windows won't recognize it anymore).

Important Note : The driver we're going to use integrates totally with Windows, so it modifies important system files configuration, which means if the installation fail, it can destroy your system, please do have a backup of Windows (like a restore point) and a copy of your personal data. There should be no problem with the latest version, this is just in case something wrong happen.

Download Ext2fsd and install it following the default installer settings, your computer should then reboot...
Once your computer is back on, go to the "Start menu"and launch Ext2Fsd > Ext2 Volume Manager.

Important Note 2 : If you're a Windows/Linux dual-boot user then you should have Linux partitions on your hard drive.
I absolutely recommend you not to mount them ! Ext2fsd can read/write ext2/3/4 partitions according to it's documentation, but write mode is totally unstable with ext3/4 partitions.


Go to Tools > Service Management and apply my settings : http://talk.maemo.org/attachment.php...1&d=1347118033

You can now use ext2 on Windows !

Macintosh, as I haven't one I can't write a tutorial for OSx users, but you should look on the web for a tutorial about MacFuse and fuse-ext2.
If you do have a Macintosh and successfully installed ext2 support in Mac OSx, please share the tutorial you used, I will add it.



STEP 2 - ext2Manager



Follow the step mentioned on the official thread page to install it.



STEP 3 - MyDocs in ext2



I am assuming you prepared everything, your N9 should be charged, your computer turned on, and you should have backed up your all personal data from your N9.

First, I recommend you to follow those step with a clean N9 to be sure everything will happen as I expect it should happen. From now please flash your device using comprehensive reset method. This will erase all the data on your phone.

Once, the flashing process is done, please enable "developer-mode" under Settings > Security > Developer mode, your N9 will reboot, after this go back to the same menu and install "Utilities".

Then connect your device to your computer via the usb cable.
Select "Mass storage" mode.
Backup all the content of your storage memory, to a folder named "backup_orig" on your computer, be aware of ".filename" hidden files on Unix based systems, do not forget to copy them.
Now please unmount/eject your N9 from the OS only. Do not remove the usb cable ! This is important, it blocks the N9 to mount the MyDocs partition. To be formatted, a partition needs to be unmounted with the umount command, but aegis doesn't allow the use of the umount, that's why I use the "Mass storage" trick.

Now we can begin the shell part, open the meego-terminal and enter the following commands :

Code:

devel-su
cd /etc
cp fstab fstab.orig
nano fstab

When the N9 boots the fstab file is read to determined which partition have to be mounted at bootup, and where & how each partition should be mounted, it also include the name of the filesystem used for each partition. I am not going to describe more how that file works, as they're a lot of possibility to do with it.

As we will change a partition filesystem, we need to edit this file.
From now you should see this on your screen :

Quote:

rootfs / rootfs defaults,errors=remount-ro,noatime 0 0
/dev/mmcblk0p3 /home ext4 rw,noatime,errors=remount-ro,barrier=1 0 0
/dev/mmcblk0p1 /home/user/MyDocs vfat noauto,nodev,noexec,nosuid,noatime,nodiratime,utf8 ,uid=29999,shortname=mixed,dmask=000,fmask=0133,ro dir 0 0
ubi0:var /var ubifs noauto,no_chk_data_crc,bulk_read
/dev/mmcbl0p1 is the partition that interests us, this is where the content of MyDocs is stored.

Editing the fstab file :
With the arrow keys, go to the beginning of the 3rd line, press enter, and write those two character : "# ". Now go up to the blank line you inserted and write this text : "/dev/mmcblk0p1 /home/user/MyDocs ext2 rw,noatime,nodiratime,errors=remount-ro 0 0". Do not modify anything else.

I won't explain the changes made, but here's how the file should exactly look after modification :

Quote:

rootfs / rootfs defaults,errors=remount-ro,noatime 0 0
/dev/mmcblk0p3 /home ext4 rw,noatime,errors=remount-ro,barrier=1 0 0
/dev/mmcblk0p1 /home/user/MyDocs ext2 rw,noatime,nodiratime,errors=remount-ro 0 0
# /dev/mmcblk0p1 /home/user/MyDocs vfat noauto,nodev,noexec,nosuid,noatime,nodiratime,utf8 ,uid=29999,shortname=mixed,dmask=000,fmask=0133,ro dir 0 0
ubi0:var /var ubifs noauto,no_chk_data_crc,bulk_read
Note : The file is 6 lines long, the 6th line should be empty. The 4th and 5th line as they appear on TMO are an only same line (the 4th one), this is just Talk Maemo that show them as two lines !

To save the file press CTRL + o, and then CTRL + x to close it.
Back to the terminal enter this command to check the content of fstab :
Code:

cat fstab
You can now format your device :
Code:

cd /sbin
./mkfs.ext2 /dev/mmcblk0p1
./e2label /dev/mmcblk0p1 "Nokia N9"

Please note that the second command can take a few minutes to execute.

Then reboot your N9 :
Code:

./reboot
And remove the usb cable.

Your N9 should boot using the ext2 filesystem for mass storage.
Launch ext2Manager once to fix the MyDocs folder rights, and copy back the data from the backup_orig directory on your computer, do not delete this backup directory before the end of the tutorial. (Check STEP 4 to learn about how to transfer files since you formatted your N9 in ext2).

Launch the gallery, if the default pictures doesn't show up, well it's because the program configuration (tracker-control) that indexes files has broke. Let's repair it. (I experienced this bug anytime I followed my tutorial.)

Copy the user-dirs.dirs file to the MyDocs directory from your computer. Now start the meego-terminal app and enter those commands :
Code:

cd /home/user/MyDocs
cp -f user-dirs.dirs /home/user/.config/user-dirs.dirs
rm user-dirs.dirs

Then please clear your device (Settings > Reset > Clear device). The process can take a few minutes. (Remember that it deleted every app and personal data, so please then reinstall developer utilities, and ext2Manager.)

Once it reboots, open the meego-terminal, the "Clear device" process has reformatted the mass memory into fat32, so we're going to put it back to ext2... again !
Connect your N9 in "Mass storage" mode to your computer, and eject/unmount it without removing the usb cable.
Code:

devel-su
cd sbin
./mkfs.ext2 /dev/mmcblk0p1
./e2label /dev/mmcblk0p1 "Nokia N9"
./reboot

Your N9 reboots again. Copy back the backup_orig data to your N9, start the gallery, you should see the pictures now.

You're now using a fully functional N9 with MyDocs formatted in ext2. YOU CAN NOW RESTORE YOUR PERSONAL DATA. (Don't forget to run ext2Manager first.)



STEP 4 - [Reminder] Learn how to use the "Mass storage" with any of your operating systems



Windows :

Connect the N9 in "Mass storage" mode to your computer, do what you want, disconnect the N9.


Linux Ubuntu :

- Connect the N9 in "Mass storage" mode to your computer,
- then open a terminal :
Code:

sudo chown -R yourusername:root /path/to/N9mountpoint
an example :
Code:

sudo chown -R alexis:root /media/"Nokia N9"
- do what you want
- disconnect the N9
- and launch ext2Manager.


Linux :

- Connect the N9 in "Mass storage" mode to your computer,
- then open a terminal with root rights :
Code:

chown -R yourusername:root /path/to/N9mountpoint
an example :
Code:

chown -R alexis:root /media/"Nokia N9"
- do what you want
- disconnect the N9
- and launch ext2Manager.


Macintosh :

- Connect the N9 in "Mass storage" mode to your computer,
- then use the Mac OSx utility to fix rights on a folder,
- do what you want,
- disconnect the N9
- and launch ext2Manager.


THE END.

Alexislavie 2012-09-08 13:52

Re: [HOW TO] Format the MyDocs partition into ext2.
 
2 Attachment(s)
Tips :

- Assign a fix drive letter for your N9 on Windows :
Launch "Ext2 Volume Manager" then right click your N9 partition and click on "Change Drive Letter", click on the actual drive letter, and click the button change, select the new letter and don't change any other settings, click OK. That's it.

- Accessing your N9 on Windows without an ext2 driver :
Select "Sync and Connect" mode, you'll be able to see your files, or you can use bluetooth too.

- Formatting MyDocs into ext3 or ext4 :
If you're a Linux only user you can adapt this tutorial easily by changing this command :
Code:

./mkfs.ext2 /dev/mmcblk0p1
into
Code:

./mkfs.ext3 /dev/mmcblk0p1
or
./mkfs.ext4 /dev/mmcblk0p1

and edit the fstab this way :
Quote:

rootfs / rootfs defaults,errors=remount-ro,noatime 0 0
/dev/mmcblk0p3 /home ext4 rw,noatime,errors=remount-ro,barrier=1 0 0
/dev/mmcblk0p1 /home/user/MyDocs ext3 rw,noatime,nodiratime,errors=remount-ro,barrier=1 0 0
# /dev/mmcblk0p1 /home/user/MyDocs vfat noauto,nodev,noexec,nosuid,noatime,nodiratime,utf8 ,uid=29999,shortname=mixed,dmask=000,fmask=0133,ro dir 0 0
ubi0:var /var ubifs noauto,no_chk_data_crc,bulk_read
or this way :
Quote:

rootfs / rootfs defaults,errors=remount-ro,noatime 0 0
/dev/mmcblk0p3 /home ext4 rw,noatime,errors=remount-ro,barrier=1 0 0
/dev/mmcblk0p1 /home/user/MyDocs ext4 rw,noatime,nodiratime,errors=remount-ro,barrier=1 0 0
# /dev/mmcblk0p1 /home/user/MyDocs vfat noauto,nodev,noexec,nosuid,noatime,nodiratime,utf8 ,uid=29999,shortname=mixed,dmask=000,fmask=0133,ro dir 0 0
ubi0:var /var ubifs noauto,no_chk_data_crc,bulk_read
Additional informations for advanced users only :

- Benchmarks to test the Windows ext2 driver performance :

On the left fat32, on the right ext2.
http://talk.maemo.org/attachment.php...1&d=1347230606http://talk.maemo.org/attachment.php...1&d=1347230606
Note : ext2 performance should be better than fat32 on the N9, as Meego use Linux as it's kernel.

wook_sf 2012-09-08 14:03

Re: [HOW TO] Format the MyDocs partition into ext2.
 
afaik, there's no need in clearing device, just forcing tracker-control to (re)index specific directories, worked for me many times

Alexislavie 2012-09-09 20:10

Re: [HOW TO] Format the MyDocs partition into ext2.
 
Quote:

Originally Posted by wook_sf (Post 1263133)
afaik, there's no need in clearing device, just forcing tracker-control to (re)index specific directories, worked for me many times

Well it didn't work for me...

MFaroTusino 2012-09-10 01:39

Re: [HOW TO] Format and use a ext2 MyDocs partition.
 
Thanks for the tutorial :)

I did something similar, reformatting it as fat32, 4kb. Works well, especially since ext2 isn't natively supported on Mac OSX or Windows (Main OSes used today) like you say. While you can still adapt your N9 to work, it isn't as simple as "plug 'n' play" of Fat32.

Still though, great tutorial and good on you for sharing it :) Just thought I'd add my two cents

wook_sf 2012-09-10 09:19

Re: [HOW TO] Format and use a ext2 MyDocs partition.
 
what's on-device R/W speeds?

Alexislavie 2012-09-23 15:11

Re: [HOW TO] Format and use a ext2 MyDocs partition.
 
Quote:

Originally Posted by wook_sf (Post 1264124)
what's on-device R/W speeds?

I do not know a disk benchmark utility on the N9, so I can not really test if ext2 is faster than fat32 with meego harmattan, but it seems to be, according to my experience with my N9.

ladoga 2012-09-24 17:45

Re: [HOW TO] Format and use a ext2 MyDocs partition.
 
1 Attachment(s)
Something like bonnie++ might work for testing on N9. hdparm is useless as it tests only low level, not the filesystem.

Here's a run on my N9's fat32 partition (formatting seems to get messed up here even with code tags, some lines are wrapped):
Code:

~/MyDocs/tmp $ time bonnie
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96      ------Sequential Output------ --Sequential Input- --Random-
Concurrency  1    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
rm696            2G    76  98 14759  20 10265  19  820  98 36981  25 523.6 102
Latency              352ms    1498ms    1825ms  19438us    151ms    169ms
Version  1.96      ------Sequential Create------ --------Random Create--------
rm696              -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                16    36  99 28575  96  246  98    35  97 +++++ +++    84  93
Latency              107ms  54381us  27893us    114ms  18707us  93567us
1.96,1.96,rm696,1,1348505190,2G,,76,98,14759,20,10265,19,820,98,36981,25,523.6,102,16,,,,,36,99,28575,96,246,98,35,97,+++++,+++,84,93,352ms,1498ms,1825ms,19438us,151ms,169ms,107ms,54381us,27893us,114ms,18707us,93567us
real        27m 22.81s
user        0m 4.06s
sys        21m 15.06s

Note: tracker-miner-fs process seems to go mad due to test and using lots of CPU. I'm thinking if making tmp directory inside /home/user/MyDocs accesible only for root and running bonnie with -u root:root would prevent the tracker from indexing the directory (tracker-miner-fs runs with user's priviledges) and solve this.

Garp 2013-10-29 16:02

Re: [HOW TO] Format the MyDocs partition into ext2.
 
Quote:

Originally Posted by Alexislavie (Post 1263127)
Tips :

- Formatting MyDocs into ext3 or ext4 :
If you're a Linux only user you can adapt this tutorial easily by changing this command :
Code:

./mkfs.ext2 /dev/mmcblk0p1
into
Code:

./mkfs.ext3 /dev/mmcblk0p1
or
./mkfs.ext4 /dev/mmcblk0p1

and edit the fstab this way :

.

Is this file usable for windows Paragon_ExtFS_for_Windows_2.41.exe in stead of Ext2fsd v0.51.exe

death_jax 2013-11-04 13:14

Re: [HOW TO] Format and use a ext2 MyDocs partition.
 
If after following instructions you encounter the gallery (and other media) problem, it's possible to resolve it with the Gallery Fix included in N9QTweak (Tweak KK-9), following its instructions in the associated PDF file. For me it worked instead flashing the device.


All times are GMT. The time now is 19:53.

vBulletin® Version 3.8.8