gk7ncklxlts99w1 I'm wondering why you think the "algorithmic strength" (which I'm guessing you mean entropy) would be stronger with the passphrase? I don't believe there's any throttling for LUKS. If you compare the entropy of a strong passphrase with LUKS vs a relatively short PIN on a HSM that does throttling, which one would be better? I suppose it depends on the implementation?
Entropy needs to be secret, or it isn't entropy. Assume you have a wordlist that gives 13 bits of entropy per selected word, and you completely randomly using a strong random number generation process selects 6 words from that list, your passphrase has 13*6=78 bits of entropy. The moment you write down your passphrase on a piece of paper to remember it, your passphrase has 0 bits of entropy. There is no longer any uncertainty in what your passphrase is, the attacker who finds the piece of paper knows your passphrase, and will only have to do a single login attempt to succeed.
If you put the passphrase on your security key, it is effectively written down, but behind a lock of some kind. If the attacker can break that lock, they get your passphrase.
Entropy, or algorithmic strength alone, on the other hand, give you a strong proof there are no way the attacker can break into your encryption ever that is faster than trying all 278 = 300,000,000,000,000,000,000,000 possible passphrases you might have selected. Or coercing you into revealing the passphrase.
Exploits against almost any security chip are developed within a few years from the security chip having been initially released. But with entropy, ie a passphrase alone, you can select a passphrase long enough, or with much entropy enough, for it to remain impossible to break into for any number of years into the future you want it to remain secure.