An attacker with access to the phone via ADB would be a big problem. They could install any app in any profile and grant special permissions that you can't grant from the Settings app. They'd have access to change settings without your knowledge or consent. Those are just two examples. For most people, ADB should just be off all the time.
Developer options should be disabled too. Many people change settings there which cause problems later on.