As mentioned above, it's definitely always been been like this, but as I don't don't get that many incoming calls, and because I reboot quite often when fiddling with stuff, it took me a while to spot what was going on, sorry. I've tried tweaking lots of pulseaudio settings, but haven't been able to influence this in any way. I also had the idea of killing and restarting pulseaudio after answering a call, to see if I could bring back audio during the call itself, but unfortunately, it only seems to work after the call has finished.