I am experiencing this with a Pixel 7 Pro and Pixel 9 Pro. It also affects restoring, not only backing up. However, when it happens during restores, it is silent and gives no error.
In order to migrate to a new phone currently:
1. Use seedvault to make an INTERNAL backup on the phone's internal storage.
2. Copy the backup to your USB drive
3. Set up new phone *without* selecting "restore"
4. Copy your backup from your USB drive to the new phone's local storage
5. Go to settings > system > backup > 3 dots > restore > internal storage
If you try to backup or restore from a USB drive directly, it will fail. I wasted most of my day today trying to figure this out.