Hmm. I keep mine. Right now, I've got all my (non-junk-mail) e-mail going back to about 2005. It isn't always useful to keep all that stuff, but it is sometimes handy.
And while I do use IMAP, I like the ability to cache at least some mail on my phone, as I live in an area of the world with very spotty cell phone coverage...