It did work for me when I tried to do RTP over-the-network redirection (the quality and latency were not good though, the details should be in the two years-old pulseaudio network thread). I do not have USB speakers to test setting up another alsa sink in pulse though, so if the problem is that the kernel crashes (?) no idea.