ignoramous How does SOCKS5 "harden" a VPN or act as a "killswitch" on Android?
Mullvad provides SOCKS5 proxies that are only routable when connected to a Mullvad VPN server.
https://mullvad.net/en/help/socks5-proxy
hazard On Graphene, is it possible to enforce a system-wide SOCKS5 proxy? What about a per-app SOCKS5 proxy? Obviously without using a "VPN workaround" (where apps create a VPN profile for the proxy).
You would need to use a VPN app such as RethinkDNS for this but that doesn't work with your requirement to avoid a "VPN workaround". There is no way to use the official Mullvad VPN app and global SOCKS5 or per-app SOCKS5 on GrapheneOS unless every installed app properly supported SOCKS5 connections without leaks.
You could use a Wi-Fi router that connects to Mullvad VPN. If you connect to the router with GrapheneOS, you could then run RethinkDNS to configure SOCKS5 connections but this means you would not be using Mullvad's VPN app on the GrapheneOS device.
As Klingerman01 already said, GrapheneOS has improved VPN leak blocking: https://grapheneos.org/features#improved-vpn-leak-blocking.