Have you looked at the settings used by tablet-encode http://maemo.org/downloads/product/PC/tablet-encode/? It may help you to optimize your encoder settings.