Some study says that exposure to lossy compressed music can cause hearing loss. No idea if this is true. But the sound is missing pieces and this may really strain the ear, even if you don't hear any difference.
But I still don't use much FLAC because the files are too big for me.