Jeansburger So I've dug into this a bit deeper for myself on a Pixel 7 as I had the same problem. I practically gave all possible permissions to Android Auto and it's google related dependencies. I had the Default Language set to English (UK), applied the same to keyboards and voice input and made sure to download offline voice packs both in Google and the voice synthesis apps for English (UK).
Still no joy, until... I specifically opened the Sandboxed Google Play settings page/activity, which is the Graphene OS GmsCompat app under the hood, I press the hands-free button on my car and Google Assistant now works in Android auto! Locking the screen, it will still work for a minute or two until I assume the phone signals to the GmsCompat app, "you are no longer drawing the page/activity, might as well sleep/kill the app process" and then of course "Oops something went wrong!" happens once more!
I'm convinced it's something to do with GmsCompat app sleeping/being killed when not in focus. Perhaps if it's a background service of some sort it will sort our issues? I'm not an Android dev and this is my two cents from a 30 minute gander at logs and pulling my hair out in my car haha. @GrapheneOS I can obtain more logs if it helps.