secrec I don't think it actually needs to be unlocked to reinstall the factory OS, since the keys match.
Installing the GrapheneOS key makes both the GrapheneOS key and the stock key trusted, but Fastboot mode functions for flashing partitions are disabled while the bootloader is locked.
What this means in practice is that you don't need to erase the custom key to return to stock, and the stock OS wouldn't have a verified boot warning displayed, but erasing the custom key is one step further in restoring the device to a fresh-from-factory software state, because no custom key is installed there from factory.
de0u The bootloader will trust system images signed by Google, but only once they're installed.
This isn't true. The device always trusts the stock OS verified boot key, so if it ever sees an installed OS signed with the stock key with properly installed verified boot metadata (vbmeta) it would boot it. The data on the device (both Credentials Encrypted and Device Encrypted) are both tied to the currently active key (i.e. GrapheneOS's) so the stock OS would be unable to decrypt any of the data, so there isn't a security risk (this is something I personally asked the GrapheneOS team about).
de0u Installing Google's OS over GrapheneOS requires an unlocked bootloader or would require the GrapheneOS image installer in Recovery to trust Google signatures on system images
This is true.
If someone finds a vulnerability (e.g. in System Updater) that allows rewriting system partitions, they couldn't use it to insert malicious modifications because that would be caught by verified boot, but they could supply the properly signed unhardened stock OS to “unharden” the device from GrapheneOS. But as I said above, the stock OS would be unable to decrypt any of the data. If the device reboots, the stock OS would boot up but I believe it would look the same way as after a duress PIN is triggered — it would try to boot a few times and eventually lead to recovery mode for a factory reset because it can't read any data.