After some more experimentation I got this to work properly.
With the headphones plugged in I changed the sound settings so that they were the default playback device and default communications device.
I unplugged the USB headphones, and both defaults switched to my speakers.
I plugged the USB headphones back in, and both defaults switched to the USB headphones - and Syrinscape also switched to that output.
So, no bugs in the syrinscape player - but useful information in case anybody else comes across this issue.