Holy cow I wrote a long post. You are excused for not reading it.
AelwennBZH I've seen the project account state that they're aware that some people have recurring issues with security keys and Play services but that it's not clear why. There are more than 3 regressions a year, I can tell you (but some of these might be reproducible on stock PixelOS as well).
On PixelOS, the FIDO menus from Play services look and behave slightly differently compared to Sandboxed Google Play. It looks like Sandboxed Google Play displays menus that PixelOS used to display in the past. It might be that the new menus on PixelOS are simply bundled with PixelOS and not Play services itself, or that they require privileged access. It would explain the incompatibility issues with Sandboxed Google Play, but I'm just guessing wildly here.
On top of that, you have frequent changes to upstream Chromium which requires frequent testing to discover if regressions have been introduced. I'm following the GrapheneOS Alpha/beta testing chat quite closely, but cannot recall that anyone has mentioned issues with security keys and Vanadium recently.
It's not clear how many GrapheneOS users are using security keys, but the low number of issue reports suggest that there is not a high number of users that do. Furthermore, there are other regressions in the OS with almost every update to AOSP that need to be hunted down and fixed because they will affect a significant number of users.
If I were to say that "the developers of GrapheneOS probably have a lot on their hands", I suspect that that would be an understatement. I'm not sure how many resources the GrapheneOS team wants to use in fixing all the issues with security keys – or figuring out if they are fixable. The plan is for GrapheneOS to provide native support for FIDO, so how many hours would fixing existing (and weird) issues with Sandboxed Google Play take away from that, one may wonder.
There have been cases where FIDO broke so badly that the developers prioritized fixing it as soon as possible. I recall an OS release in the alpha channel being cancelled because users reported FIDO dying. (Although a fix was later released through Gmscompatconfig instead).
It seems that security keys are especially prone to breakage.
Meanwhile, it seems pretty clear to me that Google – a FIDO Alliance board level member – is not prioritizing fixing very apparent issues with FIDO in their own software. For instance, on PixelOS it is not possible to use Play Store to add a passkey to your account, because the button for it does nothing. I was initially so surprised by this that I tested it across two different Pixel devices, and since then I have occasionally checked it again between every factory reset of my Pixel 6a (about a dozen), and after probably half a year that button still doesn't work.
Please note that I do not speak for the GrapheneOS project.