A basic PIN such as 6 random digits is only secure due to the secure element throttling. It makes no sense to disable that and use an insecure lock method that's trivially brute forced. A strong passphrase such as 6-8 random diceware words can avoid depending on the secure element throttling, but it's not in any way harmed by it. It's strictly more secure to have the secure element throttling. There's zero downside.
It doesn't make much sense to trust the CPU but not the secure element. However, if that's the threat model, it is not trusted in the way you think for this but rather provides an additional input for key derivation. All of the key derivation is done on the main CPU in the OS and then in TrustZone to finish it up with hardware bound key derivation. The wrapped key system is then used to provide handles for the disk encryption keys without the OS kernel having direct access to avoid kernel information disclosures leaking the keys. Wrapped keys do not have to be used in order to use the hardware encryption acceleration but are a security feature where the keys can be used without the OS having direct access to reduce the opportunity for leaks. As an example, it prevents an app leaking the keys via a kernel vulnerability and then that being used later. It's a very minor security feature vs. the massive security boost provided by the secure element throttling and the decent boost from hardware-bound key derivation. The only thing you're trusting is the main SoC including the CPU for this, which is unavoidable in general. You can't avoid trusting the CPU.