Taixzo, thanks for working on this. I wanted to do something like this. My idea was something like this:

Run as a daemon and load language model into memory ahead of time (not sure how much memory this takes). No UI unless specifically requested. User could press the camera button or some other trigger to call up the daemon at which time it would utter a noise to indicate it is ready to listen to your command. Then, it would be able to execute many commands via dbus calls, etc. (See: http://wiki.maemo.org/Phone_control). The list of commands available should be user-configurable, with a simple default set included. You might want to look at how "Alarmed" does this for some ideas.

I think it would be useful to think of this as "Maemo Voice Assist" as someone else mentioned, rather than an app per se.