Re: Massive interactivity improvement under high I/O load!
 

I was hoping that this would fix a problem with easy debian whereby if you install certain packages like evolution the install chokes and the phones reboots, but unfortunately t doesn't.

Re: Massive interactivity improvement under high I/O load!
 

Damn.. I knew someone was gonna call me out on that. :)

The thing is, I suspect the I/O buffer on most MicroSD cards is big enough that we don't routinely hit the situation of distributed writes causing excessive erases, because for me the streaming performance wasn't affected at all.

And since Linux doesn't know about the flash cell size anyway, presumably it doesn't do that great a job at aligning the data anyway (only good by coincidence).

I think the right answer is using a real flash filesystem like jffs2, but a flash-aware scheduler would be a great idea too.

Re: Massive interactivity improvement under high I/O load!
 

so if i have swappolube installed, should i run these commands as well?or are there specific commands that will add up to swappolube?

swappolube's swappiness was set to 30, so i gues the command on the first page can be excluded..what about the others?

Re: Massive interactivity improvement under high I/O load!
 

Your work on improving interactivity is much appreciated but I want to make sure we're not missing an important point as well -- you're using your N900 as a wireless media server with Samba and copying hundreds of megabytes at a time internally.

This behavior deserves recognition for going above the call of duty to prove the N900 is indeed a pocket computer. :D

Re: Massive interactivity improvement under high I/O load!
 

Bus speeds to the microsd might be slower, however it's hella annoying to have I/O collisions on eMMC when swapping and reading/writing off of it at the same time.

I use a second partition on my microSD for swap and set it at a single higher priority than the eMMC. Cards are so cheap, I really don't care about killing it. When the rear cover is yanked, it will halt instantly. I just don't take the rear cover off :P

I also think scheduler changing (BFS?), ramzez and a change to SLUB/SLAB/SLOB/SLQB will yield the best results. The android kids have been messing with this stuff much more, unfortunately they don't always document it. They just drop kernel **** in a ROM and push it.

Re: Massive interactivity improvement under high I/O load!
 

Thanks akpoff.. :)

Honestly my dream is to have a true laptop in my pocket with zero artificial limitations. I want to be able to travel anywhere with nothing more than a bluetooth keyboard and mouse.

Re: Massive interactivity improvement under high I/O load!
 

I think an efficient algorithm would erase blocks in the background so they are ready when needed. That way only when the pool was exhausted would a write be held up by an erase. I think JFFS2 does something like that, but have no clue whether SD cards are that smart.

At any rate, your changes do seem to help interactivity without causing the performance loss when writing large files over USB or SSH that I have seen with some of the other proposed settings that have been floating around.

Re: Massive interactivity improvement under high I/O load!
 

I've been trying to compile compache... it's brilliant on everything else I use. Not had much luck so far getting a build environment to build modules for kernel-power.

Re: Massive interactivity improvement under high I/O load!
 

What i meant is how do the results of each set of changes compare to each other, not what changes are involved in each.

Re: Massive interactivity improvement under high I/O load!
 

the last line gives me:
Is that 'cause i don't got an SD card?

Re: Massive interactivity improvement under high I/O load!
 

Yes, as I understand it, that's exactly why. Run "ls" on the "/sys/block" directory. If there's no mmcblk1 folder in there, then your device just doesn't see that piece of hardware - not having it in there at all being a logical reason for it not being able to see it.

Re: Massive interactivity improvement under high I/O load!
 

Do you know what the MMC-driver actually does? It appears to simply make the device unavailable without unmounting it, but appearances can be deceiving so maybe it does more than that.

And when you say ruin, do you mean simply corrupt the data on it by not leaving it in a clean state or do you mean electrically damage so it can't be used again? If you just mean might corrupt the data on the card, I don't see how the driver removing the device without unmounting the filesystem is any better than yanking the card.

Re: Massive interactivity improvement under high I/O load!
 

(Typing this on the FastSMSEvo keyboard in portrait mode - N900 sure has come far with so little manufacturer support.) Anyway, I'm pretty certain it unmounts it. Simply making the card/MyDocs unaccessible would be useless for the only reason that's even implemented - making sure the file systems/partitions don't get raped. Nokia execs might be stupid but their programmers aren't, mostly.

And, yes, were talking about corruption digitally, not physical damage.

So yes, that is different than making the cards inaccessible. (If cards were inaccessible but mounted, I'm pretty sure it would mess up your storage just about every time you used Mass Storage mode, without jumping through way too many hoops. Just a guess though, I am by no means an expert.)

Re: Massive interactivity improvement under high I/O load!
 

I mean electrical (or lowest level logical) damage caused by breaking power supply so a erase or flash cycle gets aborted. (possible) Result: card defect, bin it.

/j

Re: Massive interactivity improvement under high I/O load!
 

Not sure on the SLUB/SLAB/SLOB/SLQB argument, but ramzez is basically just a port of compcache. I've tried to incorporate it into the BFS kernel, but Nokia screwed around so much with the swapping code (to try to align pages) that unfortunately it's unstable, especially with the swap notify patch (even when I modified it to try to allow for their swap_remap nonsense). This can be shown up very easily by simply running two memhog's of around 200MiB at the same time - OOM is the usual result, with the occasional complete system crash. Lovely..

Re: Massive interactivity improvement under high I/O load!
 

Thanks, then I guess if the driver cuts power to the slot what it does makes sense.

Re: Massive interactivity improvement under high I/O load!
 

By the way, I meant to mention but didn't have much time to post; there is also an anti-stalling I/O patch which I successfully compiled into the BFS kernel - it's on the garage page now, although disabled by default:

https://garage.maemo.org/tracker/ind...1528&atid=5523

It should also be trivial to patch into kernel-power, if anyone is so inclined..

Re: Massive interactivity improvement under high I/O load!
 

Just out of curiosity, have you tried running that memhog test with more min_free_kbytes?

Re: Massive interactivity improvement under high I/O load!
 

Yeah me too. Still would like to figure out a way to throw it all onto a bigger display, somehow...

http://farm5.static.flickr.com/4091/...d13fe967_m.jpg

Re: Massive interactivity improvement under high I/O load!
 

I've tried this:

echo 4 > /sys/block/mmcblk0/queue/nr_requests
echo 200 > /proc/sys/vm/vfs_cache_pressure

from ur setting and it seems to me an improvment.

before i did this:

echo 4000 > /sys/block/mmcblk0/queue/nr_requests.

But I have this

echo 1250 > /proc/sys/vm/dirty_writeback_centisecs
echo 1250 > /proc/sys/vm/dirty_expire_centisecs

and

usr/bin/read-ahead /dev/mmcblk0p1 4000

and also i enabled

echo 1 > /proc/sys/kernel/sched_compat_yield

which i found a huge improvement.

I have compiled read-ahead myself. what do u think? did u tried my tools? did u ran bonnie++ before and after ur optimization? i did but i forgot the result. Anyway thank u for sharing i got a hard time with this device!

Re: Massive interactivity improvement under high I/O load!
 

You're going to need a port of NX server (or NeatX).

Re: Massive interactivity improvement under high I/O load!
 

If I apply the tweaks from the 1st Post after a few days of use/uptime, my phone will freeze/become unresponsive indefinately (i waited 15 min). Only thing that worked was shutdown by long press....
If I apply them right after boot it works but the improvement in normal use is minimal. Running Power46@250-1000@ulv

Re: Massive interactivity improvement under high I/O load!
 

Update:
After a few hours of use my phone becomes more and more unresponsive, eventually so bad that I have to reboot.
Wont use those tweaks again. Swappolube is fine....

Re: Massive interactivity improvement under high I/O load!
 

I don't have this issue. I have had a couple of gmail-randomly-logging-me-out errors in microb, and a couple of spontaneous reboots. I do not, however, know if these are caused by the tweeks yet though, as it's been far too little time to tell in my opinion, as I've had a couple of spontaneous reboots shortly before applying these tweeks.

Re: Massive interactivity improvement under high I/O load!
 

I've just tried exactly that (setting 8M as min_free_kbytes), and both memhogs got killed by oom_kill_allocating_task after around 100M (of 200), unfortunately.

Re: Massive interactivity improvement under high I/O load!
 

On buffers in microsd/emmc cards, from benchmark results I'm guessing the kingston cards I tested have about 0 buffers, and some sandisk cards behaved like they were able to handle 2 or 3 write streams to different areas of the card before speed collapses.

Some sandisk class 2 cards perform a few magnitudes faster than some kingston class 6 under specific write patterns ;-)

Re: Massive interactivity improvement under high I/O load!
 

Apparently the whole class thing is utter bs. There's no independent testing & certification process, so manufacturers are free to class their cards however they like.

And in any case, it's easy to tune to the benchmark, so I'm sure many vendors will just optimize contiguous block writes to the exclusion of all else. :(

Re: Massive interactivity improvement under high I/O load!
 

Well it's hard to optimize for anything else.

The class rating for both cards I tested were probably correct, it's just that the class rating measures something that's pretty irrelevant for use by a modern multitasking operating system.

Re: Massive interactivity improvement under high I/O load!
 

The main market target for SD/microSD are so basic that there is nothing to optimize:
- additional "read-only" storage for mobile devices: music, video, maps
- photo shooting: large file written sequencely (burst)

What we just need is a standard RAM behind a microSD connector with the smallest possible power consumption ! Does it exist already ?

Re: Massive interactivity improvement under high I/O load!
 

That would be most excellent...

Or a 32gb card with, say, 1gb of DRAM split across two partitions..

Re: Massive interactivity improvement under high I/O load!
 

Hi, I suspect this is a bug that may have began fundamentally before I subjected my N900 to these kernel tweeks. However, shortly after doing so, I developed a "symptom" in the form of the N900 crashing/rebooting spontaneously when dealing with very specific UI events - namely, selecting chunks of text with Home/End/PgUp/PgDown + Shift while the system is under medium to heavy multitasking load. My N900 has the aforementioned keys mapped to FN+Left/Right/Up/Down, so it's Shift+Fn+Arrow key.

There are other rare reboots without such presses but that's where I noticed it the most. I've been fiddling with some of the kernel parameters to see how I can get the best of both worlds - I suspect that what it is is some combination of having really low-and-easy-to-reach I/O locks, plus the RAM-heavy environment of multitasking.

To be fair I didn't originally implement all of the tweeks suggested here: I used my own marginal knowedge to try to find a middle ground between the hawaii tweaks (I don't like to call them swappolube tweaks without first pointing out that they came from hawaii) and this.

But in the meantime, I want to ask: the people who are benefitting from these mods, and you using your N900 for heavy multitasking, and if so, what kind? I/O heavy multitasking, processing-heavy multitasking, memory heavy multitasking (keeping in mind that you can and probably will be heavy on all/some of them), and how would you describe your multitasking in more plain english terms? Lots of files/data loaded into ram, as in video/music/whatever playing or heavy document/file editing, heavy web-browsing, what? Because when I play youtube videos, copy/edit files and use ftp/ssh/vnc, I am mostly reboot safe. But it's when I push the system and then just lightly use interactive UI elements that involves text (the MicroB browser being by far the primary offender), and do heavy text selection primarily, that it goes and reboots.

I'm not quite sure what's going on, and it seems that rasing dirty ratios slightly, and the vfs cache at 200, and for all I know my N900 is just dying from some hardware bug (never overclocked), but it's definitely odd, and if someone else is having spontaneous reboots after having these mods installed that has similar usecases to mine (lots of using the N900 as my primary web browsing and posting-my-rants tool), might mean something, and can help come up with better optimizations that get the majority of the benefits of this without impeding other use cases.

Re: Massive interactivity improvement under high I/O load!
 

I think that the random reboots its only because u don't have a 2nd swap space and hence the bandwith of the swap-vm-sub-system isn't as large as it can be. I did for myself a heavy uses of the device too but with the 2nd swap space and this tweaks i get a really responsive device.

Re: Massive interactivity improvement under high I/O load!
 

Hey MT,

Mine's been rock solid with the config listed here. :/ I multitask heavily all the time and reboot about once a week for some reason or another.

I will admit that I stopped striping my swap as my tests from the other thread were somewhat bogus. In real-world scenarios I found swapping to the internal storage + SD was slower as:

a) it ties up the internal storage and
b) SD card is much faster for reads and writes.

I'm now swapping exclusively to SD (9mb/sec write, 17mb/sec read) and everything's stable and snappy.

Re: Massive interactivity improvement under high I/O load!
 

I don't know about a) but this isn't true for me when I stripe swap like u have suggested in ur 1st post:

/dev/mmcblk0p3 partition 786424 0 -2
/dev/mmcblk1p1 partition 639988 88092 -1

and SD card has prio -1 and is filled with 88 MB and internal has prio -2 and filled with 0 Bytes my device feels much more responsive. My 8 GB class 6 card is most likely similar in read and write to ur card. Also I'm using readahead of only 16 and nr_request of 1-4. One explanation can be that the bigger stripped swap space affects somehow the calculation of the swap function. The other is that I randomly and accidently choose a size of the 2nd swap and the 1st space together that match the 128 kb page limit but I'm really unsure. Is ur internal swap then disabled und useless waste? Anyway I got an error in my sfdisk program when it should count from 1 to INFINITY like told in the wiki about partitioning the sd card my program count from 0.

Re: Massive interactivity improvement under high I/O load!
 

Hmm. According to that /proc/swaps you're only actually using one of the swap areas (SD). If you want them to stripe you have to set them to the same priority.

Your current setup is identical to SD swap only..

Re: Massive interactivity improvement under high I/O load!
 

O.k. it's not striping but conky is showing correct size of swap space ( 1,36 Gb ). What is this setup is called?

Re: Massive interactivity improvement under high I/O load!
 

...he just told you that setup is SD on swap only. Set your internal swap to -1 and your SD swap to 0 or set them to the same priority to "stripe" across in a more equal manner.

Re: Massive interactivity improvement under high I/O load!
 

Thanks for the answer. I told him that conky is showing correct swap space size and I've asked him what is this setup is named?

Re: Massive interactivity improvement under high I/O load!
 

Because both partitions are currently active for swapping - hence why it is showing the sum of both added together. You have the internal set at such a "low" priority that nothing gets swapped because the "higher" priority one...takes priority.

Re: Massive interactivity improvement under high I/O load!
 

Hey epitath,

There are two configurations:

What you have is like raid-0 "concat" mode. Basically, when your highest priority swap runs out of space (SD, because -1 > -2), it'll start swapping to the next highest priority swap area (internal).

Your total available swap space is the sum of these two.

This configuration is particularly useful when you have block devices of differing speeds (ie. compressed ramdisk and slow media) since you can tell the kernel to use the fast device until it's full, and then start using the slow device. It would be even more useful if the kernel could migrate infrequently used pages from fast->slow in the background, but I'm not sure if anyone has implemented it.

The other configuration is like raid-0 "striping" mode. When you have your swap areas set to the exact same priority, the kernel will interleave blocks between the two. This theoretically increases both read and write bandwidth, just as it does with striped raid-0, since the kernel can parallelize the writes, and some reads.

Like the "concat" configuration, your total available space is the sum of the two, though the "stripable" area (increased bandwidth) is equal to 2x the size of the smallest area.