Augmented reality based on gestures. It's not easy to build but not impossible. Also having just a single camera may be tricky (since only one camera can't analyze the depth of field).
Now here is how I think it works. In good light conditions, the hand color have a certain colour. The phone being pointed at a screen it can't be tricked by anything else. So it scans for objects with that color and if they find something like they search for something like this shape:
Because i think if you keep your hand like this:
holding 2 fingers instead of one, the soft gets confused and won't work anymore.
It would be nice to implement stuff like this, but also unconfortable. because you can have to stay very close to the screen. Another idea was to hang this to your neck, hock it up to a pico projector and do all that from whenever you are staying!
This is what I have found:
"Plug and Touch uses the camera in the Nokia N8 to work out exactly what you’re up to. It takes photos at 10 frames per second, with the processor in the handset keeping an eye on you. If you stay in one place for a certain time – with your finger hovering for longer than 0.5 seconds, for instance, the phone works out that you’re trying to press a button. It’s also possible to scroll, and carry out other functions all via giant actions."
