Massive interactivity improvement under high I/O load! - Page 2 - maemo.org

Creamy Goodness | # 11 | 2011-02-17, 04:07 | Report

Originally Posted by TiagoTiago View Post

How does this compare to Swappolube?

you can adjust most of these settings from swappolube, but its missing the

Code:

/proc/sys/vm/min_free_kbytes 
/sys/block/mmcblk0/queue/nr_requests
/sys/block/mmcblk1/queue/nr_requests

why couldn't you look yourself?

Mentalist Traceur | # 12 | 2011-02-17, 04:09 | Report

Originally Posted by nightfire View Post

Never heard of swappolube...

In any case, like Mentalist Traceur said, blocking I/O doesn't result in a failure, it just means the process read()s, write()s, etc.. won't return immediately, but only once there's sufficient room in the (now very small) queue.

The benefit is that some desktop application that needs to read a config file doesn't risk ended up at queue position 100, waiting for 99 I/Os to complete, because cp, tar, or samba wasn't able to burst all those writes through (write() blocked quickly).

Swappolube came from Hawaii's optimizations. Most of them are similar to yours, however they were more along the lines of reducing occurances of I/O in general, and reducing swapping when not necessary. Swappiness was set to 30, and a few other changes were made. dirty and dirty_background ratios were put really high, and he didn't reduce the queues... Eventually someone packaged those changes as "swappolube" as a script, then it got made into a GUI program. Anyway, the difference is hawaii's original mods aimed for a I/O less, I/O in efficient bursts approach, where as you seem to go for a I/O often, so that I/O doesn't build up.

That said, I personally have been fiddling to get my own combination of settings based on those original mods.What I personally want to see is as little swap as possible while I still have ram space, but I don't want it to zealously abandon swapping the way it would if swapping is made zero. I'll fiddle with this myself, let you know if it seems better than what I'm doing.

It seems more in line with what I've been trying to do, which is keep as little memory as possible from being forced into swap, while still allowing for swapping when it is 'needed', since the smaller queues and the smaller dirty ratios would help keep it from accumulating dirty pages.

I'm having stellarium downloading one of the more massive (8 of 9 I believe) catalogs right now over wifi, so I don't want to fiddle too much now, but I'll get back to you when I get that over with and have had time to write the changes to where I want them, and then rebooted.

The Following User Says Thank You to Mentalist Traceur For This Useful Post:
pelago

nightfire | # 13 | 2011-02-17, 04:10 | Report

Originally Posted by Creamy Goodness View Post

you can adjust most of these settings from swappolube, but its missing the

Code:

/proc/sys/vm/min_free_kbytes 
/sys/block/mmcblk0/queue/nr_requests
/sys/block/mmcblk1/queue/nr_requests

why couldn't you look yourself?

I would, but I'm still trying to figure out how to workaround Samba's unwavering desire to pre-allocate writes... which don't cooperate with non-sparse-file enabled filesystems like the lowly fat32.

I'll take a look at swappolube later.

epitaph | # 14 | 2011-02-17, 04:10 | Report

Originally Posted by Bratag View Post

As requested. As a side point. If this does what we think it does - then the issues with transmission essentially locking up the phone (ok really slowing it down) will be mitigated to some extent.

Code:

dirty_ratio: 95

dirty_background_ratio: 60

dirty_writeback_centisecs: 0

dirty_expire_centisecs: 0

min_free_kbytes: 2039

swappiness: 30

vfs_cache_pressure: 100

onboard_nr_requests: 128

sd_nr_requests: 128

This isn't the original values. I have others. Also OP is right it makes a huge difference but i strongly doubt that it will work with transmission which is strange enough.

Last edited by epitaph; 2011-02-17 at 04:16.

epitaph | # 15 | 2011-02-17, 04:15 | Report

Originally Posted by nightfire View Post

Exactly. I'm actually able to place phone calls, browse, and launch programs while writing at 8mb/sec! It was totally unusable before.

Can u try this with my tools? I'm just curious! In the original kernel swappiness is 100.

Last edited by epitaph; 2011-02-17 at 04:18.

nightfire | # 16 | 2011-02-17, 04:19 | Report

Originally Posted by Mentalist Traceur View Post

Swappolube came from Hawaii's optimizations. Most of them are similar to yours, however they were more along the lines of reducing occurances of I/O in general, and reducing swapping when not necessary. Swappiness was set to 30, and a few other changes were made. dirty and dirty_background ratios were put really high, and he didn't reduce the queues... Eventually someone packaged those changes as "swappolube" as a script, then it got made into a GUI program. Anyway, the difference is hawaii's original mods aimed for a I/O less, I/O in efficient bursts approach, where as you seem to go for a I/O often, so that I/O doesn't build up.

That said, I personally have been fiddling to get my own combination of settings based on those original mods.What I personally want to see is as little swap as possible while I still have ram space, but I don't want it to zealously abandon swapping the way it would if swapping is made zero. I'll fiddle with this myself, let you know if it seems better than what I'm doing.

It seems more in line with what I've been trying to do, which is keep as little memory as possible from being forced into swap, while still allowing for swapping when it is 'needed', since the smaller queues and the smaller dirty ratios would help keep it from accumulating dirty pages.

I'm having stellarium downloading one of the more massive (8 of 9 I believe) catalogs right now over wifi, so I don't want to fiddle too much now, but I'll get back to you when I get that over with and have had time to write the changes to where I want them, and then rebooted.

Yeah.. for myself, performance is less of an issue than stability and predictability. I don't mind if file transfers drop from 8mb/sec to 7mb/sec, but I can't stand waiting 5 seconds for a UI response or crashes due to deadlocks, so I'll eliminate burstiness at the cost of efficiency any day.

And for swap, it's one of those frustrating balances. We want to swap http://talk.maemo.org/images/icons/icon8.gifas much unused data out as we can (ie. X init and bootup code, shells hosting blocked scripts, etc) to free memory for stuff we care about yet don't want to swap out stuff we might want again (and in a hurry). My argument for a low swappiness factor is.. since we're under so much pressure anyway the system will not have trouble finding excuses to page out.

I wonder if it would be worth setting some binaries unswappable (ie. phone).

Last edited by nightfire; 2011-02-17 at 04:21.

epitaph | # 17 | 2011-02-17, 04:24 | Report

My tools is using mlock to lock x into ram. It seems to work. U should try it. And also hawaii is a troll.

Last edited by epitaph; 2011-02-17 at 04:27.

Mentalist Traceur | # 18 | 2011-02-17, 04:30 | Report

Originally Posted by nightfire View Post

We want to swap http://talk.maemo.org/images/icons/icon8.gifas much unused data out as we can (ie. X init and bootup code, shells hosting blocked scripts, etc) to free memory for stuff we care about yet don't want to swap out stuff we might want again (and in a hurry). My argument for a low swappiness factor is.. since we're under so much pressure anyway the system will not have trouble finding excuses to page out.

Personally, I didn't even know so much useful-only-at-boot crap sat in the ram indefinitely. If these things sit in ram after boot, can't a script be written to simply get rid of all the stuff like that that wastes memory space?

Creamy Goodness | # 19 | 2011-02-17, 04:35 | Report

Originally Posted by nightfire View Post

I would, but I'm still trying to figure out how to workaround Samba's unwavering desire to pre-allocate writes... which don't cooperate with non-sparse-file enabled filesystems like the lowly fat32.

I'll take a look at swappolube later.

nooooooooooooo not you, I'm mad at the Tiago guy because he posts like 10 useless things per day, I'm suggesting if he REALLY wanted to know, he could have looked.

The Following User Says Thank You to Creamy Goodness For This Useful Post:
nightfire

qole | # 20 | 2011-02-17, 04:37 | Report

I wonder if this would fix the problems people have been having trying to decompress the Easy Debian image. Decompressing a 2GB image in MyDocs basically brings the system to its knees and takes over an hour.

I filed a bug and nobody had any idea what was wrong...

The Following 4 Users Say Thank You to qole For This Useful Post:
kevloral, Mentalist Traceur, pelago, slender