tralala
There are kind of two approaches to handling files in Android. One is an app gets direct access to the file system, another approach is to use a file picker. Element uses the file picker. Basically, you gave element access to that file when you "opened" it.
Also, all apps can save files to certain folders even without asking for access. Element saves the recovery key file to one of those folders. Since element created that file, it can open it again. But, element cannot see any other files that it didn't create.
So, what you're describing is all expected behavior and element hasn't obtained any special access.