$ apt-cache show python-matplotlib shows a lot of information about the package (to confirm if it is installed).