Improving responsiveness under high memory load

[Cirne]

Big fat warning: if you try anything in this post, be ABSOLUTELY SURE that you can fix it if it goes wrong! You may end up needing to re-flash your N900 if you make a mistake.

I'm not sure if anyone has tried this before, but I searched and didn't find anything, so I figured I'd post it here.

I've always been a little disappointed in how the N900 handles when all its RAM is used up and it starts going to swap. It doesn't happen that often, but sometimes I have four or five browser windows open plus the email client and maybe an xterm... Typically the device pretty much stops responding, and even pressing control-backspace, the power button, or even the lock slider takes ages to work. And if I get a phone call? Sometimes I don't even get the phone window to show up before the phone stops ringing.

Taking a look using htop, it seems like the browser process and X itself are fighting over RAM-- the browser is using what appears to be around 500 or so megs of memory (most of it swapped out, obviously) and both it and X are in disk-wait as both of them page fault to work together on handling the same request. So, I've altered my N900 to make the Xorg process lock all its memory into RAM. Which means that RAM space can't ever be used by other processes, but in my experience, X swapping out is never a good thing on a workstation, and that's probably triply true on a handheld device.

The above technically-heavy description is to drive away people not savvy enough to understand what's going on here, because trying what I've done can very, very easily brick your phone. If you do try this, have some sort of way to fix your phone, like the rescue ramdisk over at the Meego site. Though I suppose just reflashing would work, too, if you don't mind having to restore from a backup.

I haven't done any serious tests on this, but it SEEMS more responsive now. How much is true and how much placebo, I couldn't say.

If you want to try this, you'll need root access. Download the attached file, untar it, and move mlocker.so into /usr/lib. Edit /etc/event.d/xomap and add LD_PRELOAD=/usr/lib/mlocker.so to the beginning of the exec line, so it should look like this:

        LD_PRELOAD=/usr/lib/mlocker.so exec /usr/sbin/dsmetool -n -8 -r "/usr/bin/Xorg $XORG_OPTIONS"

There's another copy of that file in /etc/event.replace.d/ -- I don't know why it's there, but I modified it the same way. Then reboot and pray you didn't mess anything up.

Like I said, it seems to help. It won't make your N900 any faster per se and it won't make ANY difference if you're not using up all your RAM, but if you are, things should get a little more responsive. I find that actually pressing control-backspace works better at getting me to the task switcher than half-pressing the camera button with camkeyd, but I can pretty reliably get to the switcher and close apps.

Phone app is a bit more responsive when I get a call, though not quite as much as I'd like. I'd really like to do the same thing for hildon-desktop (the window manager) and rtcom-call-ui (the phone app), and maybe systemui (the power key menu), but those are all run by maemo-launcher, so I can't do quite the same trick there. I wish there were a way to force memory page locking from outside a process in Linux, but if there is I don't know about it.

Anyway, if you've gotten this far and decide to try this, let me know if and how well it works!

[romanianusa]

Overclocking makes the device pretty responsive to me....no need to fiddling w/ RAM.

[Cirne]

Originally Posted by romanianusa

Overclocking makes the device pretty responsive to me....no need to fiddling w/ RAM.

Sure, overclocking will help in the general case. It'll make the phone more usable under general conditions, and it will make certain operations take less time. However, overclocking won't help this performance issue at all-- it's entirely based on the speed of the flash memory, which can't be sped up by overclocking the CPU.

Give it a shot for yourself: try opening up, say, eight browser windows pointing to Facebook or some other media/content-rich website. Then try opening another application. Can you even get to the application menu? Try switching back to one of your previous browser windows. How long does it take before clicking the close button actually works?

There is another point, too-- overclocking will damage your system. It's not a matter of if, it's a matter of how much. By pushing your system's limits, you decrease the life expectancy of the hardware-- and it isn't as easy to replace a burnt-out CPU in a phone as it is in a desktop machine. What I'm talking about here, if you do it correctly, won't affect your phone's hardware at all. In a certain sense, it's like marking a program as "high-priority", only instead of affecting processor time, it affects memory usage characteristics. You aren't putting more physical load on the phone's RAM, just specifying which processes get to use it first.

[phedders]

You could:
mv /usr/bin/rtcom-call-ui /usr/bin/rtcom-call-ui.real

cat <<EOF > /usr/bin/rtcom-call-ui
#!/bin/sh
LD_PRELOAD=/usr/lib/mlocker.so
exec \${0}.real "$@"
EOF
chmod +x /usr/bin/rtcom-call-ui
That should work... hmm might try it

[jakiman]

Originally Posted by Cirne

There is another point, too-- overclocking will damage your system. It's not a matter of if, it's a matter of how much. By pushing your system's limits, you decrease the life expectancy of the hardware-- and it isn't as easy to replace a burnt-out CPU in a phone as it is in a desktop machine.

I am very much interested in tuning and making the N900 faster. Both hardware and software methods. But saying that overclocking will damage the N900 with such certainty is a bit overboard. (although, I know what you are trying to say. )

Yeah, my phone even at 1GHz+ is sometimes very laggy and unresponsive when even trying to pick up an incoming call. (But I have to say, my N900 is not exactly "stock")

[gregoranderson]

Originally Posted by phedders

You could:
mv /usr/bin/rtcom-call-ui /usr/bin/rtcom-call-ui.real

cat <<EOF > /usr/bin/rtcom-call-ui
#!/bin/sh
LD_PRELOAD=/usr/lib/mlocker.so
exec \${0}.real "$@"
EOF
chmod +x /usr/bin/rtcom-call-ui
That should work... hmm might try it

Seems to work nicely for rtcom-call-ui ; mileage may vary with others.

For example, just tried it with camera-ui and we're stuck in an endless boot cycle

I predict a lot a flashing. Be careful out there people ... test on your scratchbox first.

[Cirne]

Originally Posted by phedders

You could:
mv /usr/bin/rtcom-call-ui /usr/bin/rtcom-call-ui.real

cat <<EOF > /usr/bin/rtcom-call-ui
#!/bin/sh
LD_PRELOAD=/usr/lib/mlocker.so
exec \${0}.real "$@"
EOF
chmod +x /usr/bin/rtcom-call-ui
That should work... hmm might try it

Short answer: no, it won't work.

Long answer: I tried that at first (actually, a slightly more complex version of that; you can see the remnants of it in mlocker.c). The problem is that those processes aren't actually standalone binaries. The file you mention is just a symlink to maemo-invoker. maemo-invoker, in turn, talks to maemo-loader (a daemon), telling it "please run the rtcom-call-ui application". maemo-loader then opens up /usr/bin/rtcom-call-ui.launch and executes it. Only here's the rub, and why all the hassle with maemo-loader in the first place: the .launch files aren't actually executables. Oh, they're binaries, to be sure, but they're more like shared libraries with a main function. maemo-loader loads them and transfers control over to them without ever calling exec().

You can do an ls -l /proc/*/exe to see just how many processes are running with maemo-loader as their system-recognized primary executable. So, whatever hacks I'm going to do, they'll have to be more complex and target maemo-loader itself. I certainly plan to, at some point; might even throw a renice() in there too. It'd sure be keen if all the processes on the critical path didn't have to wait for page faults or processor scheduling when you get a phone call.

[Cirne]

Originally Posted by gregoranderson

Seems to work nicely for rtcom-call-ui ; mileage may vary with others.

If by "work" you mean "doesn't crash the system", then yes. If you mean "locks pages into RAM" then no. The way to see if it's working is to cat /proc/(pid)/status and see if the locked memory counter is greater than 0. There is no easy way to get this working, sadly, see my above post.

[Cirne]

Originally Posted by epitaph

@cirne: Could you build a .deb package please? What is mlocker.so?

I might make a package out of this if there's enough interest, probably after adding maemo-loader support and some logic to automate patching the event.d files (with a heaping load of sanity-checking for good measure). mlocker.so is the library I wrote that hacks executables into locking their own memory into RAM at startup, it (along with its source code) is in the tar.gz archive I attached to the OP.

[James_Littler]

Originally Posted by jakiman

saying that overclocking will damage the N900 with such certainty is a bit overboard.

No it is completely true.
By overclocking you WILL damage the CPU faster than if you weren't overclocking in the same scenario.

No ifs, no buts.