I see, better contact the seller before purchasing then, ask them to confirm oem unlocking is available in the developer options. Thing is, a lot of listings say the device is unlocked, what they mean with that is you aren't locked to any specific carrier because the phone has been paid off, but the restriction on oem unlocking may very well still be in place.
Gee Could you maybe fill me in if there are any downsides at all using a Pixel 6 pro with GrapheneOS ?
No, all supported devices receive an equal amount of love.
Gee Peeking around in the forum, it seems to me that Pixel 4 or 5 are not really options anymore.
Indeed, their support duration has either run out already or will have sooner than later. See this table for an overview. The 6 pro still has almost 3 years of minimum support remaining so still a reasonable choice.