https://discuss.grapheneos.org/d/11111-lan-ip-access-troubles
I had the same question as you have, it's part of my post above.
It seems that android just generally blocks any connection to LAN when "block connections without VPN" is on. Doesn't matter if there is an actual valid VPN connection available or not. I haven't found an answer to that behaviour yet, my own guess, and I may be wrong, is that VPN apps use tunnels, which have an own LAN IP address (you can see them with eg. a terminal), so I assume that "block connections without VPN" categorically blocks any connection to LAN IPs with no exception to avoid bypassing the actual VPN (=tunnel). As said, it's my own guess and it might be wrong.