Massive interactivity improvement under high I/O load! - Page 5 - maemo.org

leetnoob | # 41 | 2011-02-17, 15:32 | Report

Originally Posted by qole View Post

I wonder if this would fix the problems people have been having trying to decompress the Easy Debian image. Decompressing a 2GB image in MyDocs basically brings the system to its knees and takes over an hour.

I filed a bug and nobody had any idea what was wrong...

I was hoping that this would fix a problem with easy debian whereby if you install certain packages like evolution the install chokes and the phones reboots, but unfortunately t doesn't.

nightfire | # 42 | 2011-02-17, 15:47 | Report

Originally Posted by joerg_rw View Post

That's not entirely correct. flash memory has massive penalty on writing randomly to different erase pages. I planned to share a link here, but have to realize I can't find the one good one. Google might help, or ask shadowJK on #maemo IRC. Basically the point is - depending on number of buffers in MMC controller - the writing to a different erase page (wich are sized like e.g 128kb) makes the controller erase a flash-page and write the dirty internal buffer to flash, then read back to buffer the new page addressed, and finally modifies the content of buffer according to the pending IO. So for a sequence like write to 0x10000, 0x50000, 0xB0000 it makes almost no difference if the individual write IO is for 1 byte or 50kB - it's the erasing and rewriting of a whole page that takes xx milliseconds, while IO to the same erase page actually only modifies the RAM buffer in controller and takes no time at all.

AFAIK a lot of elementary stuff already is mlocked, e.g rtcom-dialer-ui

What we really need is maybe a scheduler (plus kswapd) tailored to fit the exact operation mode and capabilities of the particular flash controller

/j

Damn.. I knew someone was gonna call me out on that.

The thing is, I suspect the I/O buffer on most MicroSD cards is big enough that we don't routinely hit the situation of distributed writes causing excessive erases, because for me the streaming performance wasn't affected at all.

And since Linux doesn't know about the flash cell size anyway, presumably it doesn't do that great a job at aligning the data anyway (only good by coincidence).

I think the right answer is using a real flash filesystem like jffs2, but a flash-aware scheduler would be a great idea too.

The Following 2 Users Say Thank You to nightfire For This Useful Post:
Mentalist Traceur, slender

one1002 | # 43 | 2011-02-17, 16:03 | Report

so if i have swappolube installed, should i run these commands as well?or are there specific commands that will add up to swappolube?

swappolube's swappiness was set to 30, so i gues the command on the first page can be excluded..what about the others?

akpoff | # 44 | 2011-02-17, 16:11 | Report

Originally Posted by nightfire View Post

Hey everyone,

I've been using my phone as a wireless media server for friends (with Samba + wifi hotspot). While it worked brilliantly for reads, writes were causing all kinds of weird issues, and absolutely destroyed interactivity on the phone itself.

Later, while copying several 400mb files from internal storage to my SD card, it actually froze, and rebooted. I suspect dsme was swapped out and couldn't respond quickly enough to avoid the watchdog.

Your work on improving interactivity is much appreciated but I want to make sure we're not missing an important point as well -- you're using your N900 as a wireless media server with Samba and copying hundreds of megabytes at a time internally.

This behavior deserves recognition for going above the call of duty to prove the N900 is indeed a pocket computer.

Last edited by akpoff; 2011-02-17 at 16:15.

The Following 5 Users Say Thank You to akpoff For This Useful Post:
MartinK, nightfire, qole, slender, Xagoln

hawaii | # 45 | 2011-02-17, 16:20 | Report

Bus speeds to the microsd might be slower, however it's hella annoying to have I/O collisions on eMMC when swapping and reading/writing off of it at the same time.

I use a second partition on my microSD for swap and set it at a single higher priority than the eMMC. Cards are so cheap, I really don't care about killing it. When the rear cover is yanked, it will halt instantly. I just don't take the rear cover off :P

I also think scheduler changing (BFS?), ramzez and a change to SLUB/SLAB/SLOB/SLQB will yield the best results. The android kids have been messing with this stuff much more, unfortunately they don't always document it. They just drop kernel **** in a ROM and push it.

Last edited by hawaii; 2011-02-17 at 16:26.

The Following 3 Users Say Thank You to hawaii For This Useful Post:
Creamy Goodness, joerg_rw, Mentalist Traceur

nightfire | # 46 | 2011-02-17, 17:09 | Report

Originally Posted by akpoff View Post

Your work on improving interactivity is much appreciated but I want to make sure we're not missing an important point as well -- you're using your N900 as a wireless media server with Samba and copying hundreds of megabytes at a time internally.

This behavior deserves recognition for going above the call of duty to prove the N900 is indeed a pocket computer.

Thanks akpoff..

Honestly my dream is to have a true laptop in my pocket with zero artificial limitations. I want to be able to travel anywhere with nothing more than a bluetooth keyboard and mouse.

The Following 2 Users Say Thank You to nightfire For This Useful Post:
Arrancamos, qole

bobh | # 47 | 2011-02-17, 22:07 | Report

Originally Posted by nightfire View Post

The thing is, I suspect the I/O buffer on most MicroSD cards is big enough that we don't routinely hit the situation of distributed writes causing excessive erases, because for me the streaming performance wasn't affected at all.

I think an efficient algorithm would erase blocks in the background so they are ready when needed. That way only when the pool was exhausted would a write be held up by an erase. I think JFFS2 does something like that, but have no clue whether SD cards are that smart.

At any rate, your changes do seem to help interactivity without causing the performance loss when writing large files over USB or SSH that I have seen with some of the other proposed settings that have been floating around.

phedders | # 48 | 2011-02-17, 22:41 | Report

Originally Posted by lma View Post

On the subject of swap, do take a look at the ramzez and Diablo Turbo threads. On Diablo at least, ramzswap is the single most effective way to improve responsiveness when memory-starved (which is pretty much always if you're actually using the device).

I've been trying to compile compache... it's brilliant on everything else I use. Not had much luck so far getting a build environment to build modules for kernel-power.

TiagoTiago | # 49 | 2011-02-17, 23:23 | Report

Originally Posted by Creamy Goodness View Post

you can adjust most of these settings from swappolube, but its missing the

Code:

/proc/sys/vm/min_free_kbytes 
/sys/block/mmcblk0/queue/nr_requests
/sys/block/mmcblk1/queue/nr_requests

why couldn't you look yourself?

What i meant is how do the results of each set of changes compare to each other, not what changes are involved in each.

TiagoTiago | # 50 | 2011-02-17, 23:38 | Report

the last line gives me:

Code:

-sh: cannot create /sys/block/mmcblk1/queue/nr_requests: nonexistent directory

Is that 'cause i don't got an SD card?