I'd argue you can use the Facebook API to get avatars, because I already do in Hermes. The point is that most people probably don't have three "Dave Smith"s as Facebook friends and in their Contacts.

Hermes currently does:

• Facebook
• Twitter

Future possibilities:

• Google Contacts (Mail for Exchange doesn't retrieve images, I'm told)
• gravatar.com
• maemo.org ;-)