Apart from radio channels blocking when using same band there is some hardware cause.
BT & WLAN on nxx0 share the same antenna as well as same RF amplifier.
That means it has to be switched quite fast. I don't say it's impossible to switch accurately (we can hear something anyway even using WLAN) but I think this could be a part of our problem.
I say our since I also experience poor audio quality (I'm not talking about audio's frequency range) - drops and clicks.
In fact It was well under my wife's acceptance level. I've had to switch the unit back to wired headset.