It's quite normal for Xorg to be one of the most CPU heavy processes on the phone, All GUI shits need to go throu it ..if you want to meassure the real CPU usage of Xorg when idle , do it with screen off, with 'top' from remote box. (so Xorg is not used) ....while your screen is on, Xorg will always use cpu time.
Short anwser -> 5-10% CPU usage while GUI shits are moving around (like conky for example) is quite normal.