Assuming you have the USB-C port set to the default "Charging only when locked", if it gets disconnected while locked, it will become "charging only" until the device is unlocked again. It could be that you have lint on your port, or the adapter is damaged, which causes temporary disconnections. And because of that setting, the port sets to "charging only", disabling audio output, when one of those small interruptions to the connection occur. Make sure the port is clear of lint and that the adapter isn't damaged.
You could also set the USB-C port to always on, but you would expose yourself to an adversary with access to your device being able to use the port when trying to gain access to your data. So I'd rather clean the port and replace the adapter if it's broken.