No. Its access cannot be blocked that way.
I actually checked the AOSP code quickly. Just did a Ctrl + F search. Turns out they require this access only to see if you're wearing bluetooth headphones. I didn't read more of the code to see why, but I'm guessing it's so an alert doesn't blow your eardrums out.
As far as I can tell, the app doesn't have any access to change settings, so you can just turn off Bluetooth when it's not in use and the app can't see anything.