Active Topics
-
Full linux distros on Sailfish OS (254)
to SailfishOS by qoh - 10 hrs, 16 mins ago -
Porting apps to Leste (29)
to Maemo 7 / Leste by Arno_11 - 6 days, 9 hrs ago - more...
|
2010-01-28
, 18:55
|
|
Posts: 16 |
Thanked: 0 times |
Joined on Jun 2009
|
#72
|
Originally Posted by titan
This seems like the best solution to me. Perhaps now that the camera is working with non FAT partitions the loopback file is not necessary. Although mass storage mode might not behave without it.
I think the cleanest solution would be:
/emmc - single ext3 partitions on the eMMC mounted on /emmc
/emmc/opt - directory for addons managed by system
/emmc/home - user directories
/emmc/loopback/ - directory for loopback file incl. MyDocs FAT file
If you really want to boot multiple systems you could create an appropriate
symlink to opt.<version> during boot.
Also, would ext2 be preferable to ext3?
What would be useful would be a guide or a script to set this up, I wouldn't really trust myself to do this properly. The closest thing I can find is this.
One other thing somewhat related. How trustworthy are programs with regards to putting large files in /opt? is it likely that one will ever run out of space on the 256mb NAND flash? Might mounting /usr on the internal card be necessary?
Thanks.
|
|
2010-01-28
, 21:34
|
|
Posts: 946 |
Thanked: 1,650 times |
Joined on Oct 2009
@ Germany
|
#73
|
correct, the loopback file is only useful for mass storage mode (or until recently, to keep the camera happy).
regarding ext2: I guess the Nokia engineers have carefully evaluated both fs
and decided that the journaling in ext3 is not really an issue, but rather an advantage
as there is no fsck before mounting (see bugs #7572, 7708)
you can find HOWTOs in this thread and the other one, e.g http://talk.maemo.org/showpost.php?p...1&postcount=66
I agree we should definitely compile the information on the wiki.
I'm really busy with other stuff ATM and I'd appreciate if someone
could collect the links to all relevant posts and summarize the possible solutions
(e.g. default, single /home with optional loopback, swapped vfat+ext3, MMC partitioning,
what should be exported per USB, $HOME layout and integration of MyDocs).
Then we could modify the ke-recv and upstart packages to support all our solutions
and hopefully propagate them to Nokia, so that it works out of the box.
It is quite likely that you will run out of space on NAND if you install additional applications. The optification is still work in progress and I think there
are several design flaws which need to be fixed first (e.g. /var/lib/dpkg or apt should
be moved, /tmp is too small).
Moving /usr completely to eMMC is AFAIK not an option as it may contain some essential components which are required before mounting /home (but I may be wrong).
regarding ext2: I guess the Nokia engineers have carefully evaluated both fs
and decided that the journaling in ext3 is not really an issue, but rather an advantage
as there is no fsck before mounting (see bugs #7572, 7708)
you can find HOWTOs in this thread and the other one, e.g http://talk.maemo.org/showpost.php?p...1&postcount=66
I agree we should definitely compile the information on the wiki.
I'm really busy with other stuff ATM and I'd appreciate if someone
could collect the links to all relevant posts and summarize the possible solutions
(e.g. default, single /home with optional loopback, swapped vfat+ext3, MMC partitioning,
what should be exported per USB, $HOME layout and integration of MyDocs).
Then we could modify the ke-recv and upstart packages to support all our solutions
and hopefully propagate them to Nokia, so that it works out of the box.
It is quite likely that you will run out of space on NAND if you install additional applications. The optification is still work in progress and I think there
are several design flaws which need to be fixed first (e.g. /var/lib/dpkg or apt should
be moved, /tmp is too small).
Moving /usr completely to eMMC is AFAIK not an option as it may contain some essential components which are required before mounting /home (but I may be wrong).
Originally Posted by Jophish
This seems like the best solution to me. Perhaps now that the camera is working with non FAT partitions the loopback file is not necessary. Although mass storage mode might not behave without it.
Also, would ext2 be preferable to ext3?
What would be useful would be a guide or a script to set this up, I wouldn't really trust myself to do this properly.
One other thing somewhat related. How trustworthy are programs with regards to putting large files in /opt? is it likely that one will ever run out of space on the 256mb NAND flash? Might mounting /usr on the internal card be necessary?
|
|
2010-01-31
, 19:35
|
|
Posts: 7 |
Thanked: 4 times |
Joined on Jan 2010
|
#74
|
I'm interested to know what effect on performance it has, to repartition and format the eMMC.
My understanding is that the eMMC works like a memory card. A while ago I tried formatting one to ext3, it worked at approximately half the original speed. Then when I formatted it on a PC back to FAT32, it was still about half the original speed. It was only when I used the manufacturers flash formatting software did it go back to full speed.
The clusters of sectors in the filesystem should be aligned with the physical blocks of the flash memory. Otherwise it results in multiple accesses, slowing everything down.
The flash formatting software put the start of partition at an offset from the start of the card memory, and had a different size for the FAT data when compared with the Windows partitioning & formatting.
It is necessary to get both the address of the start of the partition, and also the address of the first cluster of data (which follows the metadata) correct, to get the full performance.
This is a link I found to an explanation of cluster alignment:
http://www.hjreggel.net/cardspeed/cs_calign.html
My understanding is that the eMMC works like a memory card. A while ago I tried formatting one to ext3, it worked at approximately half the original speed. Then when I formatted it on a PC back to FAT32, it was still about half the original speed. It was only when I used the manufacturers flash formatting software did it go back to full speed.
The clusters of sectors in the filesystem should be aligned with the physical blocks of the flash memory. Otherwise it results in multiple accesses, slowing everything down.
The flash formatting software put the start of partition at an offset from the start of the card memory, and had a different size for the FAT data when compared with the Windows partitioning & formatting.
It is necessary to get both the address of the start of the partition, and also the address of the first cluster of data (which follows the metadata) correct, to get the full performance.
This is a link I found to an explanation of cluster alignment:
http://www.hjreggel.net/cardspeed/cs_calign.html
|
|
2010-01-31
, 19:59
|
|
Posts: 946 |
Thanked: 1,650 times |
Joined on Oct 2009
@ Germany
|
#75
|
thanks. that's a useful observation and the same problem also affects new harddisks
with blocksizes > 512bytes (AFAIK the flash erase block size is typicall 64K).
I think the the eMMC has a different wear leveling strategy than consumer flash media
but I may be wrong.
I've just uploaded a bonnie++ port to extras-devel so that you can
start performing benchmarks with differents fs and devices (NAND, eMMC, MMC).
with blocksizes > 512bytes (AFAIK the flash erase block size is typicall 64K).
I think the the eMMC has a different wear leveling strategy than consumer flash media
but I may be wrong.
I've just uploaded a bonnie++ port to extras-devel so that you can
start performing benchmarks with differents fs and devices (NAND, eMMC, MMC).
Originally Posted by jom
I'm interested to know what effect on performance it has, to repartition and format the eMMC.
|
|
2010-01-31
, 22:37
|
|
Posts: 946 |
Thanked: 1,650 times |
Joined on Oct 2009
@ Germany
|
#76
|
some results of my bonnie++ benchmarks
(after disabling tracker "tracker-processes -k"
and using "/usr/sbin/bonnie\+\+ -n0 -f -d." on different media)
I had to disable the file creation benchmarks as they were extremely slow on fat.
fatpart=eMMC partition, fatMMC=on class 6 SD card, xLOOPy=x filesystem in loop file mounted on y on eMMC partition
ext2LOOPext3=easydeb image in home
raw results:
please replicate the tests on your device. there is noticeably variance.
Last edited by titan; 2010-01-31 at 22:46.
(after disabling tracker "tracker-processes -k"
and using "/usr/sbin/bonnie\+\+ -n0 -f -d." on different media)
I had to disable the file creation benchmarks as they were extremely slow on fat.
Code:
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ext3 472M 7355 12 5320 8 19294 20 727.3 9
nand 472M 24352 18 14912 45 30583 80 5573.1 99
fatpart 472M 9483 15 7197 13 17879 22 970.0 17
fatmmc 472M 10255 14 5511 10 14908 17 743.0 9
fatLOOPext3 472M 2137 3 3066 5 15120 15 198.5 3
ext2LOOPfat 472M 7651 8 5936 8 18320 18 167.1 2
ext2LOOPext3 472M 7947 7 6059 8 18309 18 419.7 7
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
ext3 16 5259 69 +++++ +++ 3684 29 4841 62 +++++ +++ 4112 35
nand 16 4506 49 8280 46 4466 44 5177 54 +++++ +++ 3923 65
ext2LOOPext3=easydeb image in home
raw results:
Code:
ext3,472M,,,7355,12,5320,8,,,19294,20,727.3,9,16,5259,69,+++++,+++,3684,29,4841,62,+++++,+++,4112,35 nand,472M,,,24352,18,14912,45,,,30583,80,5573.1,99,16,4506,49,8280,46,4466,44,5177,54,+++++,+++,3923,65 fatpart,472M,,,9483,15,7197,13,,,17879,22,970.0,17,,,,,,,,,,,,, fatmmc,472M,,,10255,14,5511,10,,,14908,17,743.0,9,,,,,,,,,,,,, fatLOOPext3,472M,,,2137,3,3066,5,,,15120,15,198.5,3,,,,,,,,,,,,, ext2LOOPfat,472M,,,7651,8,5936,8,,,18320,18,167.1,2,,,,,,,,,,,,, ext2LOOPext3,472M,,,7947,7,6059,8,,,18309,18,419.7,7,,,,,,,,,,,,,
Last edited by titan; 2010-01-31 at 22:46.
| The Following User Says Thank You to titan For This Useful Post: | ||
|
|
2010-02-01
, 15:00
|
|
Posts: 7 |
Thanked: 4 times |
Joined on Jan 2010
|
#77
|
I tried this test with a new N900 with the default partitions as shipped from the factory.
The first time I ran the tests I got poor results, so I rebooted my N900, logged in with putty via ssh, and run the tests while the display was off.
I ran /usr/sbin/bonnie\+\+ -u root -n0 -f -d
with the following directories:
ext3-emmc /home/tmp/
nand /tmp2/
fatpart-emmc /home/user/MyDocs/tmp/
fatmmc-sd /media/mmc1/tmp/
ext2LOOPfat /.debian/tmp2/
The results are here:
raw data:
The microSD card I was using is an old 2GB I had lying around, which is very slow as you can see. What was the model & make of your microSD card?
It looks like you've lost some performance on the fat partition of the eMMC.
The first time I ran the tests I got poor results, so I rebooted my N900, logged in with putty via ssh, and run the tests while the display was off.
I ran /usr/sbin/bonnie\+\+ -u root -n0 -f -d
with the following directories:
ext3-emmc /home/tmp/
nand /tmp2/
fatpart-emmc /home/user/MyDocs/tmp/
fatmmc-sd /media/mmc1/tmp/
ext2LOOPfat /.debian/tmp2/
The results are here:
Code:
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
ext3-emmc 472M 8139 14 5514 9 18712 22 1024 14
nand 472M 23076 29 16016 49 33616 87 3756 98
fatpart-emmc 472M 13574 19 7266 13 19705 24 1095 11
fatmmc-sd 472M 4911 7 2752 4 9678 10 413.7 5
ext2LOOPfat 472M 10761 13 6649 9 18499 17 150.7 2
Code:
nand,472M,,,23076,29,16016,49,,,33616,87,3755.9,98,,,,,,,,,,,,, ext3-emmc,472M,,,8139,14,5514,9,,,18712,22,1023.6,14,,,,,,,,,,,,, fatpart-emmc,472M,,,13574,19,7266,13,,,19705,24,1095.4,11,,,,,,,,,,,,, fatmmc-sd,472M,,,4911,7,2752,4,,,9678,10,413.7,5,,,,,,,,,,,,, ext2LOOPfat,472M,,,10761,13,6649,9,,,18499,17,150.7,2,,,,,,,,,,,,,
It looks like you've lost some performance on the fat partition of the eMMC.
| The Following User Says Thank You to jom For This Useful Post: | ||
|
|
2010-02-01
, 15:15
|
|
Posts: 946 |
Thanked: 1,650 times |
Joined on Oct 2009
@ Germany
|
#78
|
it is probably a good idea to setup a standardized procedure for the benchmarks. How about:
a fresh, reflashed device with 51 firmware, with only ssh and gainroot installed,
no applications started, only Wifi, tests run after reboot as user via ssh.
run "tracker-processes -k" and the for each fs use
in addition to the results also report SD card specs, partition scheme and disk usage
for each tested partition.
mine is a Transcend MicroSDHC 8GB Class 6 SD card.
it would be interesting to see a comparison of ext2 vs. ext3 partition
to check whether journaling has a large effect.
a fresh, reflashed device with 51 firmware, with only ssh and gainroot installed,
no applications started, only Wifi, tests run after reboot as user via ssh.
run "tracker-processes -k" and the for each fs use
Code:
/usr/sbin/bonnie\+\+ -n0 -f -d pathtofs
for each tested partition.
mine is a Transcend MicroSDHC 8GB Class 6 SD card.
it would be interesting to see a comparison of ext2 vs. ext3 partition
to check whether journaling has a large effect.
Originally Posted by jom
I tried this test with a new N900 with the default partitions as shipped from the factory.
It looks like you've lost some performance on the fat partition of the eMMC.
|
|
2010-02-01
, 21:20
|
|
Posts: 7 |
Thanked: 4 times |
Joined on Jan 2010
|
#79
|
Titan, I think what you suggest will give the most accurate results. When I run the test I also thought about reflashing it, but I didn't want to lose my installation. So I ran 'top' to see if there were any busy processes first.
I did run tracker-processes -k
I've remembered now from my previous tests on a flash card a while ago that formating FAT32 to a cluster size of 32KB significantly improved performance. I think most flash cards and USB sticks are formatted to this (though I'm not sure how to see the parameters on the N900). ext 2 & 3 only support a maximum of 4KB per cluster (8 sectors).
I think ext3 journalling has an effect on writes only.
When you did your write testing, what parameters did you use?
I did run tracker-processes -k
I've remembered now from my previous tests on a flash card a while ago that formating FAT32 to a cluster size of 32KB significantly improved performance. I think most flash cards and USB sticks are formatted to this (though I'm not sure how to see the parameters on the N900). ext 2 & 3 only support a maximum of 4KB per cluster (8 sectors).
I think ext3 journalling has an effect on writes only.
When you did your write testing, what parameters did you use?
|
|
2010-02-02
, 08:29
|
|
Posts: 946 |
Thanked: 1,650 times |
Joined on Oct 2009
@ Germany
|
#80
|
I also did use a "fresh" installation, so my results may be biased as well (but all processes were sleeping).
IIRC you need to set -S15 for mkfs.vfat
I used standard vfat options with -F32 and ca. 520MB free on a 2GB FAT partition.
journalling may also affect parallel read+write operations.
BTW, I realized that the NAND performance may be the best-case scenario
as it is a compressed FS and bonnie only writes zeros.
We should also benchmark copying from /dev/random (worst case) and from large typical files (e.g. libqt) from RAMdisk.
IIRC you need to set -S15 for mkfs.vfat
I used standard vfat options with -F32 and ca. 520MB free on a 2GB FAT partition.
journalling may also affect parallel read+write operations.
BTW, I realized that the NAND performance may be the best-case scenario
as it is a compressed FS and bonnie only writes zeros.
We should also benchmark copying from /dev/random (worst case) and from large typical files (e.g. libqt) from RAMdisk.
Originally Posted by jom
Titan, I think what you suggest will give the most accurate results. When I run the test I also thought about reflashing it, but I didn't want to lose my installation. So I ran 'top' to see if there were any busy processes first.
I've remembered now from my previous tests on a flash card a while ago that formating FAT32 to a cluster size of 32KB significantly improved performance. I think most flash cards and USB sticks are formatted to this (though I'm not sure how to see the parameters on the N900).
When you did your write testing, what parameters did you use?
You need it for normal export partition for computer when connect by usb. additionally without modification you will have problem to access files which was copied from computer.
in my script for change vfat to ext3 when you connect to comuter in mass storage mode partition will be unmounted on n900. After diconnect it will be mounted back.
p.s. For ext2 you will need little bit modify ke-recv. in my patch i add only ext3.
and look like kamera and ke-recv will be fixed in next update because bugs for them get nokia's internal bug tracking number...
Last edited by mikhmv; 2010-01-24 at 17:46.