Virulent I understand that, assuming the secure element is working as intended, a 6 digit pin will take over a thousand years to crack on average assuming a random pin.
This is not true. An exploit that totally undermine the security the secure element provides will be found way before then. I would not expect the secure element to be able to throttle login attempts at all anymore after 5-10 years from phone original release date.
So if you worry about your phone being taken, and need to be sure they cannot ever access your data during your life time or even within the next 10 years, follow @treequell advice and use a 6-8 word diceware password. That way, you are no longer dependent on the secure element at all for security.