obviously to start pulseaudio you need a valid config.
When your config has a non-existing ALSA audio card, then quite probably PA will quit and get respawned by dsme until dsme thinks it's enough and reboots system.

Have you checked the usual places for reboot reason (/proc/bootreason, var/lib/dsme/stats/ dsme error logs, syslog)?

I still think you don't want to redirect standard PA audio output. Rather you should find a way to run your mediaplayer with a very private unique PA that uses USB ALSA audiocard.
Alas I'm not a PA expert, as I hate this soundsystem and the concept it follows.

/j