Recently, when I put my carrier's SIM card in, I noticed that a new app had been installed automatically
It wasn't installed. The "SIM Toolkit" app is part of AOSP, and is part of GrapheneOS. It is a frontend for STK applications on your SIM card. The app is disabled by default, and only is enabled if your carrier has applications which expose support for STK apps to the OS from your SIM card.
I find it rather amusing that even in this OS, where security is the utmost priority, the carrier can override all the protection in place and install their own untrusted code on my phone.
See my prior comment.
what was the rationale behind not letting users uninstall carrier apps. Better yet, why are they installed on their own in the first place?
See my prior comment.
Is there actually any assurance that the carrier won't install a RAT? (other than the fact that I am not a person of interest to the state)
Carriers cannot install apps into GrapheneOS, We don't ship with any support for OMA-DM apps (device management system which can be used by carriers, it is used on Verizon in the US) and no 3rd party carrier apps (e.g. the Verizon app) at all are built into the OS. unlike the stock OS, which does ship them (this extends to the OMA-DM apps too to be clear).
Now for the highly unlikely (really must stress that part) but interesting part. The carrier (or someone with enough privileges at your carrier) could run arbitrary code on your SIM card (To make it super clear: said code is not running in the Android OS) via JavaCard applets (which is how you implement STK applets). This is how SIM cards work, it is part of the trust you place into your carrier. STK doesn't give any access into the Android OS to the carrier. An adversary could in theory gain access one way or another to run their malicious code on your SIM card, but they would be confined to the SIM card, so they can only "touch" the baseband, and the data/information passing through it. This though is pretty much the same data/information the carrier (or someone with enough privileges at your carrier) could also be able to obtain without needing the ability to run arbitrary code on your SIM card. All of the above applies to eSIMs and iSIMs too.
Hope that is clear. Let me know if you have more questions.