apt-get upgrade should be perfectly safe. Generally speaking, all that's not being automatically updated by App Manager are libraries - the newer versions are not required by any applications, so they don't get automatically updated. There should be minimal risk in updating these though (there's a small chance that the new version has dropped some functionality the app requires). apt will tell you exactly what it's doing, so just watch out for anything being removed/downgraded - otherwise you should be okay.