maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Brainstorm (https://talk.maemo.org/forumdisplay.php?f=47)
-   -   [Under consideration] Remove 256MB limitation of the rootfs partition in the N900 (https://talk.maemo.org/showthread.php?t=43833)

wongdg 2010-06-02 12:34

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
If you read the brochure on Samsung OneNand, the 256MB model in fact is the lowest end model of the family. The siblings include 512Mb, 1Gb, 2Gb, 4Gb, and the largest 8Gb.

Has anyone contemplating swapping out the 256Mb for the 8Gb. I believe they should be pin compatible, just some fine art of soldering needed. Can such a trick be coordinated? and how much? Any possibility to get Nokia to do it for a bunch of us eager user to go full linux.

I would most prefer to have a full debian phone with a decent battery life.

Wilson
Supporter of Debian linux

cb22 2010-06-02 12:39

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I recall reading earlier in this thread that the NAND is located inside the OMAP3 core...

pelago 2010-06-02 15:09

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by cb22 (Post 695707)
I recall reading earlier in this thread that the NAND is located inside the OMAP3 core...

That is correct. It won't be possible to change without changing the processor, which is practically impossible.

EDIT: Turns out that I wasn't correct here.

Matan 2010-06-02 15:17

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by cb22 (Post 695707)
I recall reading earlier in this thread that the NAND is located inside the OMAP3 core...

No, that is wrong. The nand is soldered directly on the OMAP, a method called PoP (package on package), but they are still different packages.

wmarone 2010-06-02 15:18

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by cb22 (Post 695707)
I recall reading earlier in this thread that the NAND is located inside the OMAP3 core...

Quote:

Originally Posted by pelago (Post 695970)
That is correct. It won't be possible to change without changing the processor, which is practically impossible.

Actually, being a Samsung part it is located in the same package as the 256MB of SDRAM, which is soldered on top of the processor. Theoretically you COULD replace it, but it would require access to some rather expensive hardware, and it is rather hard to buy such chips individually.

Also, wongdg, you may need to check your numbers as they typically sell flash memory in MegaBITS and not MegaBYTES, so divide by 8 to get the maximum capacity; Samsung's 8Gb (1GByte) didn't exit sampling and enter production until well after the N900's design cycle ended.

The chip in the N900 is a 2GigaBIT device, the one in the DROID and N1 are 4GigaBIT (512MByte,) and nothing I know of uses the 8GigaBIT (1GByte) chips.

mrrg 2010-06-20 14:59

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Hello Ive been reading through some posts here..and this solution might be stupid call it solution D

So rootfs is to small 256MB and moving it to the /home partition in large flash will cause user data to disappear at reflash.

So what about repartitioning the large flash reducing the size of the 27GB partition creating a new root partition on the large flash lets say 1GB.

Now reflashing this partition should not overwrite any user data ?

Perhaps /dev/mmcblk0p1 could be shrunk and then remove the empty primary partition /dev/mmcblk0p4 and add it again just after /dev/mmcblk0p1 and make this the new rootfs.

Dunno how to do the last thing though.


Ohh I just realized solution D2

You could scrap the swap partition 768MB and use this as a new root partition.

Swap can then be added as a swap file (instead of a partition) inside the /home directory.

Bundyo 2010-06-20 16:12

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
If you read through this thread you would have known that the very slow internal flash is no option. ;)

dbrodie 2010-06-20 17:48

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Well what about getting bcache running on this device? It still is very alpha from what i've seen and we have the problem of different kernel versions (maybe on a meego kernel?), but that should solve the problem, right?

stefanmohl 2010-06-21 19:42

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I think that the solutions from this thread boil down to four basic types:

1) Put most of the system in the big flash and manually selected files/directories in the fast flash (Brainstorm solutions #1 and #5)
2) Keep optification, but clean it up so that packages end up completely in /opt, without touching rootfs (currently, optification places some files in rootfs and some in /opt) (Brainstorm solution #2)
3) Put the whole system in the big flash and use the fast flash as an automatic cache (Brainstorm solution #3)
4) Don't run Maemo, run Debian (or something else) in a chroot

Of course, a fifth option is that no solution is needed:

5) Optification is not a problem, the systems work well and we have enough space in the rootfs as it is.

Some personal opinions of what I prefer after the discussions in this (and other) threads:

Out of (1) and (3), I believe mostly in (1), since I don't believe that automatic cache schedules will be precise enough to make me happy. However, to make (1) work, we need to figure out precisely what files/directories really need to be in fast flash. As an illustrating anecdote: If you move /lib/dsp to the big flash, the phone will still work fine, but CPU usage will go through the roof. Titan has done some nice work linked from Brainstorm Solution #5

(2) looks nice and clean, but I suspect that there will always be apps that need the performance of the fast flash, so reserving the fast flash only for the system is a bit too much of a limitation

I don't like (4) very much; I want to run Maemo, not Debian. If some Debian apps can easily be made to work under Maemo, that is a bonus. I don't mind if they are a few releases old (like back in Etch). Heck, I am still running XP on my Windows box at work, and that is a lot older! The main point is, I don't want to be running two different systems at the same time.

In my opinion, (5) is plain wrong, as seen from the large number of users that continue to have problems with limited rootfs space. This fact alone makes (5) wrong and is my main reason for starting this brainstorm.

Please go to the Brainstorm and vote for your preference!

mrrg 2010-07-02 21:10

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I guess I was thinking that it should be like Brainstorm solutions #1 and #5
but this way you would get a partition free for it...

Also some people say the difference in speed between the flashes do not differ so much... or I am just plain crazy.. haaahh!!

egoshin 2010-07-07 20:30

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I did my own little research with flash access times. I did it via multiple usage of command

time dd if=src of=dest bs=64k count=3000

where 'src' and 'dest' were combinations of /dev/mtd5, /dev/mmcblk0p3, /dev/mmcblk1p2 (I set swap here) and /dev/null or /dev/zero.

The results are (time in seconds, measured with stock PR1.2 kernel):

MTD5 (flash-on-top-of-CPU, root FS):
READ -
wall time = 9.93,
sys time = 9.79
battery current=1400

/dev/mmcblk0p3 (regular N900 swap space, part of 32GB internal flash):
READ -
wall time = 9.87
sys time = 2.57
battery current = 1223, repeated=1870
battery current=1252
WRITE -
wall time = 11.27
sys time = 2.28

/dev/mmcblk1p2 (a partition on micro SD, class 6):
READ -
wall time = 14.94
sys time = 3.44
battery current=977, repeated=1620
battery current=1100
WRITE -
wall time = 19.23
sys time = 3.10

----------------------------------------------

So, the interpretation - the current usage of MTD5 as root file space is useless from many point of view - the Read performance is the same as internal 32GB flash BUT (!) - CPU usage is overtop!

Actually, I strongly suspect that MTD5 access performance is limited by CPU performance (due to compression in UBIFS?) - wall time == system time.

The move of root FS from MTD5 into 32GB may benefit not only in space but CPU usage and battery life. We need only switch off the disk block caching for file content - it seems that if kernel finds disk block in cache it copies block from that cache and that consumes MORE energy than DMA read from flash chip: repeated reads take more battery current than first one.

After root FS moved to 32GB flash the MTD5 can be used without compression for first order swap, it would greatly accelerate overall N900 performance (today swapout-swapin of huge behemoths like modest/gst-video-renderer may take up to 10secs during call answer). Or use it as file cache (my proposal #3).

EDIT: Battery current updated with more accurate measurement (N900 set offline). No significant fluctuation anymore.

shadowjk 2010-07-07 23:02

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
What unit is your battery current figure in? How did you measure it?

The big difference for nand vs emmc isn't in sequential read/write.. Whereas the emmc can do something like 4-8Megabyte/s write per second sequentially, the built-in flash translation layer can not at all deal with random write (such as swap, /opt and MyDocs), and the speed for 4K random writes degrades to a few hundred kilobytes.

The OneNand has no FTL, so we can manage it ourselves, and we have a 600MHz cpu and 256 megs of ram, plenty of room to do something intelligent. As a result, the onenand with ubifs is a few magnitudes faster at serving IOs from a multitasking demand-paging operating system.

Reads are same magnitude in speed, as long as there are no writes interleaved.

egoshin 2010-07-08 00:45

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by shadowjk (Post 744277)
What unit is your battery current figure in? How did you measure it?

sleep 15
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
time dd if=$1 of=/dev=null bs=64k count=3000
cat /sys/class/power_supply/bq27200-0/current_now

(bq27x00_power.ko module is from Matan's collection)


Quote:

The big difference for nand vs emmc isn't in sequential read/write.. Whereas the emmc can do something like 4-8Megabyte/s write per second sequentially, the built-in flash translation layer can not at all deal with random write (such as swap, /opt and MyDocs), and the speed for 4K random writes degrades to a few hundred kilobytes.
Swap is not an issue at all - it is read and written in big blocks and it is located NOW in 32GB flash (non NAND!). Also I tested files on /home/user Ext3 - the file read speed is about the same - 19MByte/sec. And reading uncachable /usr/lib/locale/locale-archive from UBIFS gives 18MByte/sec. I don't see a difference besides CPU resource and battery usage.

Short files from /etc and gconfig can land in kernel cache pretty soon. And we may combine file caching in MTD2 for that - it would get the advantage of fast short access if UBIFS implementation works fine for short files.

Besides of that it is possible to use not ext3 but some more flash-friendly FS - LogFS is released in May, for exam.

Quote:

The OneNand has no FTL, so we can manage it ourselves, and we have a 600MHz cpu and 256 megs of ram, plenty of room to do something intelligent. As a result, the onenand with ubifs is a few magnitudes faster at serving IOs from a multitasking demand-paging operating system.
May be. But I don't see it now - my battery depletes too fast and I know the reason now. Personally, if there is a choice I prefer to have some slow disk access but much less CPU expenses, in my long UNIX experience with old and slow disk equipment it handles file access very well because of kernel buffer cache, especially in multitasking. But I don't think we would have a slow access.

Quote:

Reads are same magnitude in speed, as long as there are no writes interleaved.
I have a home server which is configured to have root FS read only besides some critical files for X11 which are symlinked to /var and the whole /var is located in Generic Compact Flash. It is done to prevent disk spinning and works very well, at least faster then N900 :)

So, don't worry about writes... well, gconf should be killed or replaced because it writes on any bump. But we can symlink that stuff into OneNAND.

shadowjk 2010-07-08 17:53

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Just try gpodder on your N900 with a moderately big datbase (say 100-200 podcasts). When you start a download, the 8 megabits coming from 3g to MyDocs, the 4k random write/read swap activity by the 8mbit io pressure making linux swapout to increase cache, and the random read/write to ext3 to update databases results in half-minute long stalls..

My home server also runs on flash. With ext3, the squid proxy server stalls for half a minute at a time when the flash slows down. With nilfs2, which by accident is quite performance optimized for flash, there are no stalls and doing a backup of the squid cache dir is also about 8 times faster than with ext3.

Matan 2010-07-08 18:14

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by shadowjk (Post 744277)
What unit is your battery current figure in? How did you measure it?

We don't know the units, as they depend on the resistance of a specific resistor on the board. But the values are linear, so they are useful for such comparisons.

Quote:

The big difference for nand vs emmc isn't in sequential read/write.. Whereas the emmc can do something like 4-8Megabyte/s write per second sequentially, the built-in flash translation layer can not at all deal with random write (such as swap, /opt and MyDocs), and the speed for 4K random writes degrades to a few hundred kilobytes.

The OneNand has no FTL, so we can manage it ourselves, and we have a 600MHz cpu and 256 megs of ram, plenty of room to do something intelligent. As a result, the onenand with ubifs is a few magnitudes faster at serving IOs from a multitasking demand-paging operating system.
But the swap is actually on the eMMC, so I don't really get your point. In normal activity there is very little write access to the root filesystem.

Do you have some numbers for "a few magnitudes faster ..."? It sounds too incredible to be true.

Quote:

Originally Posted by egoshin (Post 744130)
So, the interpretation - the current usage of MTD5 as root file space is useless from many point of view - the Read performance is the same as internal 32GB flash BUT (!) - CPU usage is overtop!

Actually, I strongly suspect that MTD5 access performance is limited by CPU performance (due to compression in UBIFS?) - wall time == system time.

Using root file system on eMMC is slower, even though the hardware is better, as those benchmark show, due to concurrency with swap access also on eMMC.

The best options I can think of are:
  • Use a microsd card with root filesystem on eMMC and swap on uSD (or vice versa)
  • Use root on eMMC and swap on NAND.

Both options should give better performance than the current setup and give a larger (configurable) root file system size.

The downside to first method is probably increased power usage due to uSD card.

The downside of the second is limiting of swap to 250MB rather than the current 768MB.

egoshin 2010-07-08 19:22

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by Matan (Post 745152)
The best options I can think of are:
  • Use a microsd card with root filesystem on eMMC and swap on uSD (or vice versa)
  • Use root on eMMC and swap on NAND.

The downside to first method is probably increased power usage due to uSD card.

My measurements show the DECREASED power usage with uSD (see previous message). It is one of latest uSD Class 6 cards.

Quote:

The downside of the second is limiting of swap to 250MB rather than the current 768MB.
I tested the two swaps for single system, it works - just add another swap partition and kernel puts it as "secondary" priority swap. It works fine with automatic extension to the 2nd layer swap.

But I am afraid the usage of NAND has a disadvantage in battery performance. I observed high CPU usage for reading just raw 64KB data blocks from /dev/mtd5 and it seems that NAND performance is actually limited by CPU. Plus higher battery usage.

It may be because of block device emulation (not used in UBIFS) but this layer should be used for swap operations too.

So, it may have a sense to symlink or file-cache small files to NAND + UBIFS. For small file access it may be a good trade-off (Battery vs performance).

shadowjk 2010-07-08 23:57

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Caching takes care of all our concerns regarding small file access and so on, and also swap. However, this is of course only possible if we have enough RAM to cache things in ;-)

The few magnitudes claim comes from something called write amplification. The native flash block size is some 256kilobytes. Writing a 4k sector (as the OS often does) causes the emmc/sd to erase a new 256kb block from a pool of spares, (if there isn't one already erased available), read in 256kb, modify 4k of it, write it to the new block, put the old block into the spare pool.

Thus, a 4k write request from the OS was turned into a 256k read, 256k erase and 256k write. 256/4 is a factor 64 already :)

On the emmc/sd, 256k physical blocks are linearly mapped to 256k logical blocks.

With nand we can do it smarter, while the erase size is huge 256k, an erased block doesn't need to be completely written in one go. We can write 4k and fill in the rest later. We have the cpu to map blocks of 4k nonlinearly onto the 4k flash, so that even if the filesystem wants to update a table at the start of disk, a file entry in the middle, and some data elsewhere, we turn it into 3 writes into the same block.
This is roughly what modern SSDs also do, and they have ARM cpus and 64 or more megs of ram to do it, which explains why there are no sd cards doing clever stuff.

Now, normal filesystems are optimized for rotational media. There all kinds of non-sequential access becomes slow. So, they operate on the idea that data is read more often than written, and thus spend alot of effort on placing the data optimally, so that related data is close together. This is exactly the opposite thing you want to do on a flash based device.

nilfs2 and logfs, however, are very good for performance on emmc/flash :)
(and ubifs, but that can only be used on nand)

So yes, if we can use LogFS, and if we can have swap on its own device, or someone writes a new loop block device specifically to optimize swap on flash, having non-media system files on emmc would be doable..

Without logfs though, I suspect we get more of all sortsof "stuttering" people cmplain so much about even now :)

geneven 2010-07-12 21:11

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
So I have a 32 gb I think class 10 card. Any chance I could use it for this? How, exactly?

stefanmohl 2010-07-13 00:35

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I have been wondering about that myself, could anyone confirm if 32GB cards work without problems? The phone is officially only rated for 16GB cards AFAIK.

Edit: found a thread that verifies that 32GB microSD cards work:
http://talk.maemo.org/showthread.php...t=32GB+microSD

The_Solutor 2010-07-14 02:00

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I faced a similar problem running a regular linux installation on top of a RS-MMC (damn slow, and too little for a regular linux distro)

I tried almost every solution available (unionfs, aufs, symlinked directories and so on) and finally I tried BTRFS with compression enabled and SSD mount opition.

The result is simply amazing, the system is incredibly fast, the bootime is almost a quarter than the one over a regular ext3 FS, and there is no space wasted as happens with the readonly fs overlaid by writable fs, trough aufs or unionfs.

So I vote for completely ignore the 256MB NAND and use a partition on 32 GB eMMC formatted with compressed BTRFS for the whole FS.

Eventually the 256 MB nand can be used for caching purposes, as suggested before.

The flashing problem can be eventually solved flashing not directly the emmc but the 256MB nand flash as happens now.

A script on the first boot can format the emmc partition and then copy everything on it (adjusting the configuration files when needed), so no worry for the user data on the vfat section.

egoshin 2010-07-16 20:29

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I am able now to run Maemo 5 PR1.2 from internal eMMC with working cell phone after some modifications. So, UBIFS is not used after boot and root is located in eMMC.

No root FS limitations, anything works... well, one small problem still exists but there are workarounds.

First impression performance: something different. Sometime it can be a little slow but in many cases it has definitely a faster reaction. Of course, I set swap into external MMC.

Looking now into battery performance. Without GSM it is of course very good, but now I have it with working GSM/3G/WiFI, so - still collecting measurements.

stefanmohl 2010-07-17 01:53

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
Your battery results are going to be very interesting. I had a strange experience with moving /lib/dsp to the eMMC. Granted, it might have been a fluke, but when I did so, the CPU got stuck at 500MHz and never went below. When I moved the directory back, it fell back to normal.

egoshin 2010-07-20 18:34

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
OK, couple of day passed and my N900 works fine from eMMC (swap on uSD).

Now I am ready to say that there is no notable difference in power consumption. Actually, it looks like it is a little better (around 20%) but it is in error margin - I rebooted multiple times and it takes battery resource.

Speed. Sometime it is worse but most of it during reboot until I attach uSD swap instead of /dev/mmcblk0p3. However, in day-to day clicks it looks better during changing menus etc.

No problem with phone call receiption - I means, no delays, no sound distortion etc.

No problem with e-mail - it looks like it works a little better: I have a big e-mail account (thousands mails) and it handle it's retrieval now without any problem. In vanilla boot from UBIFS it can sometime hangs (lack of memory) or do it long time.

No any problem with video playback. At least a couple of my video files play smooth.

Of course, no problem with browser or RSS feed.

I even tested google latitude - it works fine too.

It is possible to note an existence of some intermediate windows during starting some applications (like e-mail) - it blinks a fraction of second and disappears before I can read it's content during e-mail application start. But overall reaction is the same.

Two problem still exist with MCE - it doesn't work correctly with power button and doesn't finish "screen slide unlock" (MCE is a buggy application anywhere). But side slider works fine. And there is some another problems with power-button menu like some woodoe needs to be done to see it and switch OFF position doesn't work. All of them are easy to workaround and for now it is not a stop-gap.


But biggest advantage - 2GB root FS, of course - no problem with installing new applications...

egoshin 2010-07-21 22:09

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
1 Attachment(s)
Yet another day passed and I am able to compile and build the whole kernel in my N900 - full power development cycle.

Wall time - 3h 1m 21s.
eMMC space - 943MB.

I did a couple of phone calls in a middle without compilation interruption :) Skype was working too - anything as usual. Delayed reaction on button press was expected and I observed it but it was reasonable.

N900 was warm but not very. It took around 3/4 of battery resource.

AlMehdi 2010-07-22 21:27

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I wish i had the guts and knowledge like you Egoshin. Good work! ;)

egoshin 2010-07-29 15:34

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I prepared tool scripts and published it here - http://talk.maemo.org/showthread.php?t=59374

After applying this scripts you can have a 2GB (or 32GB - depending from your /home partition size) root file system.

stefanmohl 2010-08-01 00:48

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
This is really great Egoshin! You should package the scripts and put them in extras-devel as soon as some people have tried them out. Apart from easy availability and official endorsement (as it moves into extras-testing, etc), it will also let un-optified packages simply be made dependent on your package and people could still install them. Oh, and perhaps find a name that is a bit more descriptive than M32GB... ;-)

appnss 2010-08-01 08:19

Re: [Waiting] Remove 256MB limitation of the rootfs partition in the N900
 
First of all, sorry if I am posting about something that have already been covered. I have had my N900 for less than a month so I may have missed some older threads/posts even as much as I have tried to read everything related before posting.

At the first moment I saw the rootfs limitation on N900 I thought about the simple yet elegant way it is solved in OpenWRT:

A root filesystem (squashfs) + an overlay with mini_fo (jffs). This gives a readonly basic install plus an additional space for modifications.

I have been holding from trying before I read all I can about this subject in the forum, considering that if it hasn't been done yet must be for a good reason.

Anyway, before going any deeper into this subject, I would like to understand what is the reason because this option was ruled out.

I have only been able to to find a reference in a several months old post from @SR:


Quote:

Originally Posted by @SR (Post 518339)
I've been experimenting with aufs, unionfs and mini_fo for a while to mount ext3 partition on top of internal memory. All of them have problems.
One common problem was that both of the file system partitions needed to be writable. It is because of the complex algorithm when renaming non empty folder from read only file system (aufs page has a good explanation on that). For example application managed did not work because of this problem. So mini_fo did not work at all.

What is the reason that both fs need to be writable? The idea would be to have "original" root fs mounted as read only and a writeable overlay using mini_fo the same way is done in OpenWRT.

Reads would still be faster as mini_fo doesn't have a big impact performance (needs to be confirmed in this case though) and writes would be directly into the overlay, which wouldn't be significantly slower than writing to the emmc.

About renames... well, that would be obviously much slower but... I don't see a reason for almost any rename of the original binaries much less a non-empty original dir.

Also, simply not mounting, or clearing the overlay would leave a perfectly clean fresh device. And mounting different overlays allows for some easy multiple system.

P.S.: Perhaps Egoshian way has better performance, but anyways I still I would like to understand why this simple solution was ruled out.

DrWilken 2010-08-01 08:32

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
I don't quite get it.

Why would You want to use the 2GB "/home" partition as Your / partition ?

Instead of using the 256MB NAND + 2GB /home You end up with "only" 2GB in total (+ possible trouble upgrading to next PR)...

On PR1.2 I haven't had any problems with / being filled up. I do, however, believe that /home (containing /home/opt) is a bit restricted in size (2GB). It can easily be filled up by installing lots of games etc.

I'd rather just shrink the FAT32 filesystem by let's say 3GB and add them to /home.

This is what my disk usage looks like:

Code:

N900:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
rootfs                228M  156M  68M  70% /
ubi0:rootfs          228M  156M  68M  70% /
tmpfs                1,0M  76K  948K  8% /tmp
tmpfs                256K  88K  168K  35% /var/run
none                  10M  72K  10M  1% /dev
tmpfs                  64M  4,0K  64M  1% /dev/shm
/dev/mmcblk0p2        2,0G  1,6G  308M  84% /home
/opt/pymaemo/usr/lib/python2.5
                      2,0G  1,6G  308M  84% /usr/lib/python2.5
/opt/pymaemo/usr/share/pyshared
                      2,0G  1,6G  308M  84% /usr/share/pyshared
/opt/pymaemo/usr/lib/pyshared
                      2,0G  1,6G  308M  84% /usr/lib/pyshared
/opt/pymaemo/usr/share/python-support
                      2,0G  1,6G  308M  84% /usr/share/python-support
/opt/pymaemo/usr/lib/python-support
                      2,0G  1,6G  308M  84% /usr/lib/python-support
/dev/mmcblk0p1        28G  3,0G  25G  12% /home/user/MyDocs
N900:~# du -sh /*
980K        /bin
0        /boot
12K        /bootmenu.sh
0        /cdrom
72K        /dev
6,3M        /etc
0        /floppy
4,6G        /home
0        /initrd
24M        /lib
0        /media
0        /mnt
1,3G        /opt
0        /proc
544K        /root
2,0M        /sbin
0        /srv
0        /sys
0        /syspart
76K        /tmp
288M        /usr
35M        /var

So IF I were to repartition I'd rather see something like this:
/ 256MB ubifs on NAND
swap 768MB/1GB on eMMC
/home 1GB on eMMC *
/opt 4GB on eMMC *
(/usr 512MB on eMMC *)

* Or You could just make /home 5GB (though I'm not that big a fan of the /opt symlinking - it's just ugly). I'd also like to get rid of the /opt/pymaemo symlinks. Maybe we could create a /usr filesystem too (~512MB).

And the rest for the FAT partition.

EDIT -> Guess that *solution* is discussed here.

javispedro 2010-08-01 15:05

Re: [Waiting] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by appnss (Post 771883)
What is the reason that both fs need to be writable?

Because a Nokia-blessed solution would have needed to accommodate for over the air updates, which require a writable rootfs.

Of course, for your own use you can completely ignore updates.

appnss 2010-08-02 02:06

Re: [Waiting] Remove 256MB limitation of the rootfs partition in the N900
 
Quote:

Originally Posted by javispedro (Post 772179)
Because a Nokia-blessed solution would have needed to accommodate for over the air updates, which require a writable rootfs.

Of course, for your own use you can completely ignore updates.

If that's the ONLY reason, IMHO this option was wrongly ruled out.

For all purposes, the rootfs (not the original, but the original + overlay combination) is perfectly writable. Only for a phisical reflash some steps would be needed to do it correctly, but again, in the current configuration a phisical reflash would also leave /opt somewhat out of synch.

In fact, a process to prepare for a reflash / resynch can be made before the update using the mini_fo approach. I had to do it many times with OpenWRT and never had a problem.

I have read about a watchdog proccess that detects delays in writing during boot and hangs, which could be of some concern, but that also seems to have been solved by Egoshin.

I would leave things at they currently are on nokia side, making now a change is nonsense and would lead to some troubles, but I think the mini_fo modification is perfectly compatible and would add the advantages of:

1- Avoid hang on boot.I have read in some older posts that completely filling rootfs prevents the system from booting. I hope that there's already a workaround for this in PR1.2 or some other third party util (bootmenu?). The workaround would be simple anyways (a 10MB file that gets removed on boot and reacreated after suceess), but I am not sure if my current configuration is protected against that and even after the effort being done in optifiying some apps may fail and cause some havoc.

Ie: Fring for N900 creates a HUGE log on /. I only used it a couple of times for testing and the other day I say a 25MB log file in /, which I have already symlinked to another place as a workaround.


Nokia-N900-51-1:/# ls -al /fring.log
lrwxrwxrwx 1 root root 14 Aug 2 03:29 /fring.log -> /max/fring.log
Nokia-N900-51-1:/# ls -al /max/fring.log
-rw-r--r-- 1 root root 22531688 Jul 14 15:40 /max/fring.log
Nokia-N900-51-1:/#


Luckily, being a log file, its real occupation in ubifs is much less or otherwise it would have nearly filled it.

Also, I have many binaries and libraries directly moved from easydebian chroot to main os which are not EXPECTED to need to write anything to / but one never can be SURE. (Yeah, It may be wrong to do that, but for me it gets the job done and no problems YET).

2- For developing, it is the SIMPLEST way to have different customizations on boot and making sure you have a perfectly fresh system is just a matter of mounting an empty overlay on top of original rootfs.

In my tests, I have confirmed that using a loop interface even with an image file stored in /home/user/MyDocs adds almost NO noticeable overhead. So I don't expect mini_fo do it either. Also I have seen no noticeable differences between ext3 and FAT fs. The bottleneck is always emmc speed... unless you start playing with a truecrypt image file (10 times slower in writing, 2 or 3 in reading) which rules it out for anything except selective storage.

Well, if there are no other reasons against this method, I think it is time for testing... I see unionfs kernel module is included with titan kernel, but I don't find mini_fo... is there a package somewhere for the missing modules?

I would appreciate any other suggestion or something I should take into account before trying.

Thanks!

appnss 2010-08-02 05:13

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
In the meantime I am trying to decide which would be the best fs for the overlay.

Jffs2 seems a good choice in this case, and it also seems to offer some control for compression settings (perhaps no compression is best as N900 doesn't have much space limitations, unless it noticeable reduces write times) and also for setting when garbage collection may be done. I like that it allows for atomic operations and that it is flash friendly.

Garbage collection might be posponed until on charger AND idle, providing the overlay is adequately oversized.

Have anyone done any benchmarks of jffs2 or any other flash friendly fs in N900?

My only intention is to have an overlay for /, with no changes to the rest of the system. The idea is just to have an oversized rootfs just in case **** happens, and also for having multiple overlays for different purposes. All the rest (/opt etc) being unchanged.

For /opt lack of space I am just using a combination of additional loop mounted images and symlinking which works ok for me... I don't see any issue there.

pelago 2010-08-02 08:58

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
appnss, I wonder if your ideas are worth chatting with the MeeGo team, e.g. stskeeps or similar mentioned at http://wiki.meego.com/ARM/N900

javispedro 2010-08-02 09:39

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
You can only use jjfs2 over the mtd layer, and note that UBI is "kinda" its direct successor.

appnss 2010-08-02 20:52

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
@pelago

Probably the only idea that is worth trying is the mini_fo overlay. Which I would have already tried if I had found mini_fo.ko for titan's kernel.

I am downloading the SDK and will try to make a compatible module... Should be easy, but it's been a very long time since last time I had to recompile a kernel myself. I hope not to break anything :P

@javispedro

Then I guess ext3 would be the most reasonable choice, at least for now.

appnss 2010-09-01 18:18

Re: [Under consideration] Remove 256MB limitation of the rootfs partition in the N900
 
An update about the mini_fo solution I proposed:

After enabling native compilation I discovered that mini_fo isn't updated anymore. The only update/patches available are those from the OpenWRT project which hevily relies on it.

mini_fo as it was doesn't compile against kernel 2.6.28 so the only way (that I know of) would be to backport mini_fo from OpenWRT and I haven't had the time to do so.

Anyways, I cracked my lcd screen in the meantime I had time to think about it. Now I think the better approach is to just leave maemo as it is, perhaps increasing the size of /opt partition or adding additional image files to increase it and move/symlink stuff over there.

And have another chroot'ed environment, a maemo one (in contrast to easydebian, etc), in which to have a second system, including a native compiler with which to compile every needed programs without having to care about optifiying. It works perfectly and the underlying original maemo keeps unharmed.

For me, it's the way to go, and it covers any need I can think of.


All times are GMT. The time now is 08:34.

vBulletin® Version 3.8.8