jacobb this is likely due to the blocking of network connections outside the VPN. GrapheneOS has been gradually working to stop various leaks outside of the VPN which occur on other android operating systems.
This will likely work for you if you turn off the 'block connections without VPN' toggle under Settings > Network > VPN
If it is important for you to minimize the possibility of leaks outside the VPN it would be best to use another method to transfer files between devices. Could use Bluetooth or USB storage. Possibly some kind of service that allows file sharing via connections through the VPN, although for some people this may allow an observer to make an unwanted connection between the different network locations.